-- Logs begin at Thu 2022-06-16 17:38:15 UTC, end at Thu 2022-06-16 18:51:49 UTC. -- Jun 16 18:13:16.827206 dv-cinder-813229-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 18:13:16.837642 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 18:13:16 2022] *** Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: nodename: dv-cinder-813229-10-iscsi Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: machine: x86_64 Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: clock source: unix Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: pcre jit disabled Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: detected number of CPU cores: 4 Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: current working directory: / Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: detected binary path: /usr/bin/uwsgi-core Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: your processes number limit is 47680 Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: your memory page size is 4096 bytes Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: detected max file descriptor number: 2048 Jun 16 18:13:16.840510 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: lock engine: pthread robust mutexes Jun 16 18:13:16.842046 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: thunder lock: enabled Jun 16 18:13:16.842046 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 18:13:16.842046 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 18:13:16.884009 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: Python main interpreter initialized at 0x5630f35789c0 Jun 16 18:13:16.884009 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: python threads support enabled Jun 16 18:13:16.884009 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: your server socket listen backlog is limited to 100 connections Jun 16 18:13:16.884009 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: your mercy for graceful operations on workers is 90 seconds Jun 16 18:13:16.884396 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 18:13:16.884473 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: *** Operational MODE: preforking *** Jun 16 18:13:16.884736 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: *** uWSGI is running in multiple interpreter mode *** Jun 16 18:13:16.884736 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: spawned uWSGI master process (pid: 80136) Jun 16 18:13:16.884943 dv-cinder-813229-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 18:13:16.886377 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: spawned uWSGI worker 1 (pid: 80146, cores: 1) Jun 16 18:13:16.886377 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: spawned uWSGI worker 2 (pid: 80147, cores: 1) Jun 16 18:13:16.886529 dv-cinder-813229-10-iscsi devstack@keystone.service[80136]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 18:13:22.475725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:13:22.475725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: from cryptography.utils import int_from_bytes Jun 16 18:13:22.476993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:13:22.476993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: from cryptography.utils import int_from_bytes Jun 16 18:13:22.479066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:13:22.479066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: from cryptography.utils import int_from_bytes Jun 16 18:13:22.479576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:13:22.479576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: from cryptography.utils import int_from_bytes Jun 16 18:13:22.529674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 18:13:22.529890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 18:13:22.530059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] command line args: [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 18:13:22.530347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 18:13:22.530532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ================================================================================ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 18:13:22.530797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] admin_token = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.531054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config_dir = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.531134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 18:13:22.531247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.531341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 18:13:22.531420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config_source = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.531528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] command line args: [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 18:13:22.531607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] debug = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.531788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 18:13:22.532083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ================================================================================ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 18:13:22.532212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] admin_token = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_dir = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.532943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_source = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] debug = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_config_append = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_dir = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.533920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_config_append = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.534995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_dir = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.535911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.536982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.537942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] publish_errors = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.538856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] standard_threads = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] publish_errors = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.539913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_journal = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_json = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_stderr = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_syslog = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] standard_threads = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.540936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.541013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.541200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.541337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.541405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.541464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.541603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_journal = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.541731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.541811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_json = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.541917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_stderr = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.542078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_syslog = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.542170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.542316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:13:22.542400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.542588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.542716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.542831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.542910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.543911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.544926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.545920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.546981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.547911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.548914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.549885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.550837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.551934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.552915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.553976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.554938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.555983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.556792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.557572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.557800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.557887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.557961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.557961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.558947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.559908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.560928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.561884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.562869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.563980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.564951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.565988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.566946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.external = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.567926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.password = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.token = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.568922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.external = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.569929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.password = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.token = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.570956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.571993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.572162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.572299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.572410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.573203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.573350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.573509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.573644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.573779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.573925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.574978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.575952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.576959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.577887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.578900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.579993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.580205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.580340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.580461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.580666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.580797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.580925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.581926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.582886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.583866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.584859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.585992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.586965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.587921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.588971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.589956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.590982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.591966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.592139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.592224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.592401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.592516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.592644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.592886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.593857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.594847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.595967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.596961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.597932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.598979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.599952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.600973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.601950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.602906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.603929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.604915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.605930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.606877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.607898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.608070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.608340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.608601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.608692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.608847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.608927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.609902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.610905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.611872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.612878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.613851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.614992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.615984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.616996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.617967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.618946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.619027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.619186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.619274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.619951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.620930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 18:13:22.621054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.621313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.621561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.621822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.622068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.622325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.622563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.622812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.623057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:13:22.623232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 18:13:22.738254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.738428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.738619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.740513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.740748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.740978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.747206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.749641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.775691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.776957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.777162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.777397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.831022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.832359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.880519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.880737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.881508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.881752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.949500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.949726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.949949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.950168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.950389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.950605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.951291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.951528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.951753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.951974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.952220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.952439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.974589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.974855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.974997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.975922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.976072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.976208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.976338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.976338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.987149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:22.987522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.000299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.000299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.000623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.000752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.000877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.001905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:13:23.019367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.019579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:13:23.019920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.020147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:13:23.020422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.020634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.020723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.020927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.021023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:13:23.021181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.021440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.021708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:13:23.022036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.022355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.022673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.022988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.033253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.033533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.033801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.034081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.035212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.035553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.035884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.036264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.042625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.044690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.045968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.046250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.046532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.046813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.047099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.047360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.047629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.047899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.048193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.048462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.048669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.048751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.049049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.049399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.049748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.050086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.050403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.050731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.051065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.051395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.051725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.052111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.075857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.077519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.077804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.080440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.082180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.082525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.083136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.083537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.083822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.088327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.089359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.089645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.091917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.092226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.096782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.097004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.097098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.097282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.097373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.097608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.097886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.098163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.098434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.098705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.101803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.102087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.102364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.102642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.102911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.103180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.103445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.103707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.117738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.118020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.123377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.123535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.123662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.127240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.127653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.128195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.131808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.132533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.132743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.136968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.137817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.140678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.140980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.141263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.141401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.141548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.141826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.142106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.145160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.145461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.145739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.146045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.146320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.146594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.162179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.162480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.162734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.162995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.170654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.171146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.171455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.171726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.173918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.175662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.175946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.179970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.180856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.180976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.181219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.181507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.181731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.181823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.182526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.187122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.187398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.187670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.187970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.188279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.195682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.195969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.196279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.196632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.197013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.197293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.197645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.202253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.202535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.202813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.203089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.203465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.203735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.204009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.216689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.219980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.223335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.225125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.226780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.227067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.227360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.230907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.233434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.234879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.236554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.238845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.239810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.240127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.240526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.240691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.240833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.242386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.242799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.243216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.250645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.250880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.251162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.253909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.256312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.259119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.259400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.259677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.259951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.260855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.261497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.261497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.261772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.261988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.262259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.262394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.262531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.262714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.262856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.263035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.263249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.263382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.263650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.273140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.273428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.273709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.281978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:13:23.282259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.282538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.282826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.283107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.283497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.283774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.284054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.284336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.284616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.284905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.285183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.285450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.285710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:13:23.301587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5630f35789c0 pid: 80147 (default app) Jun 16 18:13:23.305135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167da9a-8d9d-430a-a156-178034afa575 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:23.305441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167da9a-8d9d-430a-a156-178034afa575 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:23.305723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167da9a-8d9d-430a-a156-178034afa575 None None] PATH_INFO: `/v3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:23.310541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Thu Jun 16 18:13:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:23.324892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5630f35789c0 pid: 80146 (default app) Jun 16 18:13:31.302842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:31.303422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:31.303422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:31.304801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 2/2] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:13:31.313444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:31.313985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:31.313985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:31.384208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: INFO dbcounter [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] Registered counter for database keystone Jun 16 18:13:31.407071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG oslo_db.sqlalchemy.engines [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80146) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 18:13:31.413448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writer thread running {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 18:13:31.595853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG passlib.handlers.bcrypt [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80146) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 18:13:31.596322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG passlib.handlers.bcrypt [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80146) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 18:13:31.652035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:31.653988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:31.780725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1/3] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 18:13:31.957452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:32.029448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: INFO dbcounter [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] Registered counter for database keystone Jun 16 18:13:32.048632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG oslo_db.sqlalchemy.engines [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80147) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 18:13:32.054798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writer thread running {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 18:13:32.190780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2125fe-fd45-47dd-959a-c5ca7529c77f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:32.195137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:32.195825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:32.196137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:32.196420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:32.205997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:32.259088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 18:13:32.259375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482fd07-22da-4f9f-9371-95552f430a4d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:32.271123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 3/4] 10.222.0.30 () {58 vars in 1324 bytes} [Thu Jun 16 18:13:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:13:32.277937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:32.296777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097688e-0a74-4cc8-a46a-5bf172052d4c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:32.301234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:32.302379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:32.302669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:32.302949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] PATH_INFO: `/v3/projects/243115c739b84006986ba2e6345f2509` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:32.304522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] RBAC: Authorizing `identity:get_project(project_id=243115c739b84006986ba2e6345f2509)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:32.365505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 18:13:32.366162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52103ee4-d2bb-4f1d-9eb7-d2c44f6b543c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:32.371744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 2/5] 10.222.0.30 () {56 vars in 1207 bytes} [Thu Jun 16 18:13:32 2022] GET /identity/v3/projects/243115c739b84006986ba2e6345f2509 => generated 344 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:34.549753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ee00a-8b9e-4a82-8483-b4faea9c9305 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:34.550297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ee00a-8b9e-4a82-8483-b4faea9c9305 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:34.550297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ee00a-8b9e-4a82-8483-b4faea9c9305 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:34.554538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 4/6] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:13:34.561162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:34.561613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:34.561716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:34.580995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:34.582527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:34.590827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 3/7] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 18:13:34.612884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ee00a-8b9e-4a82-8483-b4faea9c9305 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:34.625417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ee00a-8b9e-4a82-8483-b4faea9c9305 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:34.629802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826e6b1-84a3-41e9-b11f-0784af40696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:34.630449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e6b1-84a3-41e9-b11f-0784af40696b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:34.630731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e6b1-84a3-41e9-b11f-0784af40696b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:34.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826e6b1-84a3-41e9-b11f-0784af40696b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:34.640300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e6b1-84a3-41e9-b11f-0784af40696b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:34.642080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826e6b1-84a3-41e9-b11f-0784af40696b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:34.650982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 5/8] 10.222.0.30 () {58 vars in 1324 bytes} [Thu Jun 16 18:13:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:13:34.656919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:34.666342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d09c5-c8c0-4a73-8f10-6e49d8ac950b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:34.670760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e671423-2631-4cb5-9209-63a24437106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:34.671645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671423-2631-4cb5-9209-63a24437106f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:34.671930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671423-2631-4cb5-9209-63a24437106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:34.672242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671423-2631-4cb5-9209-63a24437106f admin admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:34.673646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671423-2631-4cb5-9209-63a24437106f admin admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:34.675473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671423-2631-4cb5-9209-63a24437106f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:34.677082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 4/9] 10.222.0.30 () {56 vars in 1198 bytes} [Thu Jun 16 18:13:34 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:36.863646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225939a-a3c5-4aa7-8904-a97c31e86cd2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:36.864225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225939a-a3c5-4aa7-8904-a97c31e86cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:36.864225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225939a-a3c5-4aa7-8904-a97c31e86cd2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:36.865650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 6/10] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:36.871642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:36.872211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:36.872211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:36.890302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:36.891542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:36.942048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf589a-54a5-4950-8cdf-64d40b7c88df None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:36.942336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf589a-54a5-4950-8cdf-64d40b7c88df None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:36.942619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf589a-54a5-4950-8cdf-64d40b7c88df None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:36.944080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 7/11] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:13:36.949995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:36.950299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:36.950580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:36.978260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 5/12] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:36.991209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG passlib.handlers.bcrypt [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80147) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 18:13:36.991629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG passlib.handlers.bcrypt [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80147) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 18:13:37.001899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.027758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.038054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:37.039683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:37.113347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31efcda-370e-4668-93a5-a817ccc89748 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.144014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 8/13] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:13:37.148370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.149188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.149473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.149747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.161541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:37.165782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:37.173443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:37.174571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:13:37.175595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 6/14] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:13:37.182959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.267862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.275411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d417c-b8f7-4dfa-8780-0dad6f839bd8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.278868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.288852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103532-45d4-4783-b85a-c9a6d3e061db None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.314575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.315485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.315767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.316061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.324656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.325347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.325630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.325905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.326662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:37.326972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.328392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.341550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 7/15] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:37.347035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.393007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.394620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.453670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39e176-1b02-47c3-b2da-fbf226117b0f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.488064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.488975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.489259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.489534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.490731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.492434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.494056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 8/16] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:37.498041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 9/17] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:13:37.499319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.503459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.506384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.515859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.603109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b5e3-c5e7-4b2e-ada0-8b92ae11546d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.606630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f361d7f-5f68-4daf-8f5b-de53b4fb4f27 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.638349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.639154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.639435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.639709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.640954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.641728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.642410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.642510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.642792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.643066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.644001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:37.649925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.662364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.663837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:13:37.685069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:37.685971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:37.687013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:13:37.687013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:13:37.687013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:13:37.687013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:37.687013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:37.692391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:37.737958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.753830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:37.760601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cd3ef-1ccb-46d4-aef7-958cfc02a3bf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.797248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.797908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.798194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.798471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.799307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.domain.id=default, user.id=bd8fd8fbb8b9467bac357373688c6910) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:37.799648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.801120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24d004-296c-415d-814d-0fb5203e56c1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.801789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa85505-229e-49ea-9e9b-15bc38e190c9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:37.808179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1322 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/role_assignments?user.id=bd8fd8fbb8b9467bac357373688c6910&scope.domain.id=default&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:37.840304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:37.841003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:37.841293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:37.841575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:37.842337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:37.842651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:37.844134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ad244-0b9d-4fd8-9e81-19d019cf56e5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:37.863868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 11/21] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:40.598054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0699-a0b2-4fb1-91c4-572f060fed98 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.599063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0699-a0b2-4fb1-91c4-572f060fed98 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.599550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0699-a0b2-4fb1-91c4-572f060fed98 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.601743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:40.619988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb0a60-cc96-423e-8fe3-f04cce3e418b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.620999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb0a60-cc96-423e-8fe3-f04cce3e418b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.621478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb0a60-cc96-423e-8fe3-f04cce3e418b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.649160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-eebb0a60-cc96-423e-8fe3-f04cce3e418b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:40.651531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-eebb0a60-cc96-423e-8fe3-f04cce3e418b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:40.763346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 12/23] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:40.789030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e59b4-06d3-4834-9131-e7eb90a48bd3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.789647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e59b4-06d3-4834-9131-e7eb90a48bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.790129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e59b4-06d3-4834-9131-e7eb90a48bd3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.791760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 12/24] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:40.797762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96789383-3f3b-4e5f-962c-2f9b4806f3f9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.798058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96789383-3f3b-4e5f-962c-2f9b4806f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.798330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96789383-3f3b-4e5f-962c-2f9b4806f3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.811109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e59b4-06d3-4834-9131-e7eb90a48bd3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:40.826899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-96789383-3f3b-4e5f-962c-2f9b4806f3f9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:40.828139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-96789383-3f3b-4e5f-962c-2f9b4806f3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:40.882694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:40.917783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 13/25] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:40.919682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd8572-afeb-4ac2-98d0-ca5f4d5606ac None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.920244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd8572-afeb-4ac2-98d0-ca5f4d5606ac None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.920725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd8572-afeb-4ac2-98d0-ca5f4d5606ac None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.922614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 14/26] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:40.924109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bb579-d342-4880-951f-ed47c7140b5b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.924602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bb579-d342-4880-951f-ed47c7140b5b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.925079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bb579-d342-4880-951f-ed47c7140b5b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.926598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 15/27] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:40.931666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da787e-46ef-4149-ac88-c739d2ecac63 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.932340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da787e-46ef-4149-ac88-c739d2ecac63 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.932847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da787e-46ef-4149-ac88-c739d2ecac63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.958444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-24da787e-46ef-4149-ac88-c739d2ecac63 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:40.961103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-24da787e-46ef-4149-ac88-c739d2ecac63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:40.984751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e59b4-06d3-4834-9131-e7eb90a48bd3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.029504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.030770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.031339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.031817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.045422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 16/28] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:41.052068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:41.052725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.053220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.053699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:41.054786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:41.055766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:13:41.056567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 13/29] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:41.062367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.067120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:41.097886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:41.099461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:41.197565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b318e5-919e-4774-95bb-ff5663a66a95 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.205011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 17/30] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:41.205438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.263383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.264539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.264891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.265181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.266018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.267859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.274551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:41.376165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7af8-b3ea-4380-bf8b-c3c1b7da5dec None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.429664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.430985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.431467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.432535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.433827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:41.434354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.435998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.452334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 14/31] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Jun 16 18:13:41 2022] POST /identity/v3/roles => generated 225 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:41.453977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.455368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 18/32] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:41.456723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.624145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:41.634508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e23ac3-0ce4-46e6-91cc-83df6eefa6b6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.674502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:41.726817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.727658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.727947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.728261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.754749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.756394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.767008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434235db-2aa6-4b2a-b4b9-529c9bddfa1a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:13:41.789972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:41.791439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:41.792448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:13:41.792448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:13:41.792448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:13:41.792448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:41.792448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 19/33] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:41.793143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.827012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.827843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.828212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.828581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.829430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.831326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.862785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 15/34] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Jun 16 18:13:41 2022] POST /identity/v3/roles => generated 219 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:41.863692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.935521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d083b-34cc-46da-9af4-de9b368ddbe7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.976975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.978070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.978504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.978913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.983062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.984762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.987106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 20/35] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:41.996739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:42.047712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:42.054788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67482f1-688c-4506-bd00-460c46381c82 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.112821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:42.144449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.145166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.145464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.145751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.146545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:42.146866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.149113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.178044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 16/36] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:42.183971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:42.228768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36268545-ec57-4028-a551-93f9d51c1073 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.278056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d8e0-3cb9-4dc1-bb62-014bd6a826e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.279126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d8e0-3cb9-4dc1-bb62-014bd6a826e8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.279405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d8e0-3cb9-4dc1-bb62-014bd6a826e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.279679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d8e0-3cb9-4dc1-bb62-014bd6a826e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.280916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35d8e0-3cb9-4dc1-bb62-014bd6a826e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.282387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35d8e0-3cb9-4dc1-bb62-014bd6a826e8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.284020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 21/37] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:42.285397 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52900e5f-cc26-4fda-86ce-aee3c468a73a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.285689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52900e5f-cc26-4fda-86ce-aee3c468a73a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.285964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52900e5f-cc26-4fda-86ce-aee3c468a73a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.287208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 22/38] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:42.288792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc7c7-05c8-4564-a42d-b2a99f0d1a9e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.289171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc7c7-05c8-4564-a42d-b2a99f0d1a9e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.289690 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfc7c7-05c8-4564-a42d-b2a99f0d1a9e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.292068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 23/39] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:42.293779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfc7c7-05c8-4564-a42d-b2a99f0d1a9e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:42.318427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd921ee-a1b4-44f4-aec8-723a70de92b5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.358303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355a30e-fc42-4c60-b2cb-c50a3b3e4666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.358995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355a30e-fc42-4c60-b2cb-c50a3b3e4666 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.359278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355a30e-fc42-4c60-b2cb-c50a3b3e4666 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.359559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355a30e-fc42-4c60-b2cb-c50a3b3e4666 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.360329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355a30e-fc42-4c60-b2cb-c50a3b3e4666 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.361820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355a30e-fc42-4c60-b2cb-c50a3b3e4666 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.393196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 17/40] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:13:42 2022] POST /identity/v3/projects => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:13:42.394694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b448-f11e-4992-8694-efa64d7e28bb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.394995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b448-f11e-4992-8694-efa64d7e28bb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.395280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b448-f11e-4992-8694-efa64d7e28bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.412620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:42.415619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d906b448-f11e-4992-8694-efa64d7e28bb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:42.418225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d906b448-f11e-4992-8694-efa64d7e28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:42.436361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfc7c7-05c8-4564-a42d-b2a99f0d1a9e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.467800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:42.482599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.483424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.483708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.483989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] PATH_INFO: `/v3/domains/default/users/bd8fd8fbb8b9467bac357373688c6910/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.487050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bd8fd8fbb8b9467bac357373688c6910, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.489209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.540097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 18/41] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:42.540097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.540097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.540097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.566096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:42.567604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:42.674850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 19/42] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:42.675594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:42.739875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 24/43] 10.222.0.30 () {58 vars in 1361 bytes} [Thu Jun 16 18:13:42 2022] PUT /identity/v3/domains/default/users/bd8fd8fbb8b9467bac357373688c6910/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:42.741148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:42.801289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:42.810543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903c1c6-a881-4f28-93ee-1a41f41f9c85 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.850143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.850847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.851145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.851432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.852376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.853868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.864802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4359a8-613b-4443-b913-b6c2b0f09e06 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.868558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 20/44] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Jun 16 18:13:42 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:42.912776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377b54a-b98c-4895-9daf-388a7fbf52fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.914172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377b54a-b98c-4895-9daf-388a7fbf52fa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.914515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377b54a-b98c-4895-9daf-388a7fbf52fa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.914792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377b54a-b98c-4895-9daf-388a7fbf52fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.916400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377b54a-b98c-4895-9daf-388a7fbf52fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.918211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377b54a-b98c-4895-9daf-388a7fbf52fa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.922956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 25/45] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:42.929022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:43.050651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186c5b6-cb69-4e21-a808-97a89fdc8da1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:43.092354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8485371-e70f-4ad9-8f9c-a19ff04ac48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:43.093044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8485371-e70f-4ad9-8f9c-a19ff04ac48c None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:43.093337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8485371-e70f-4ad9-8f9c-a19ff04ac48c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:43.093621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8485371-e70f-4ad9-8f9c-a19ff04ac48c None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:43.094365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8485371-e70f-4ad9-8f9c-a19ff04ac48c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:43.095835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8485371-e70f-4ad9-8f9c-a19ff04ac48c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:43.124113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 21/46] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:13:42 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:44.937601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bc8ff-c81c-4864-a826-f3ed5687be63 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:44.938129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bc8ff-c81c-4864-a826-f3ed5687be63 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:44.938129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bc8ff-c81c-4864-a826-f3ed5687be63 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:44.939526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 26/47] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:44.945178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:44.945963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:44.945963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:44.963283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:44.964634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:44.966806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:45.039100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 22/48] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:45.063761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bc8ff-c81c-4864-a826-f3ed5687be63 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.083601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:45.199671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bc8ff-c81c-4864-a826-f3ed5687be63 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.242014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bfbf0-5642-4172-99bd-a5298199f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.242972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bfbf0-5642-4172-99bd-a5298199f895 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.243333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bfbf0-5642-4172-99bd-a5298199f895 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.243719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bfbf0-5642-4172-99bd-a5298199f895 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.259263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bfbf0-5642-4172-99bd-a5298199f895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-742bfbf0-5642-4172-99bd-a5298199f895 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:45.269118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:45.270247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:45.271208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 27/49] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:13:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:45.285454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.313385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57483-aab5-4130-916d-cccf12db4741 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.313681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57483-aab5-4130-916d-cccf12db4741 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.313970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57483-aab5-4130-916d-cccf12db4741 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.315525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 28/50] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:45.320405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.320699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.320987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.342288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:45.343953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:45.407884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ef37b-854c-4fee-90e3-a2b1cdad5f98 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.431050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 29/51] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:13:45.453025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.453796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.454181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.454462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.455235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:45.455419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.455575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:45.456995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:45.459493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:45.463757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:45.472861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 23/52] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:45.474991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.599419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599217-4538-4caf-889d-32bb7961b76a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.599861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45cdaf-e53e-4a29-ae7a-3de555e2d149 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.634677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.636190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.636571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.636729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.637025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.637549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.637832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.638112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.638645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:45.639342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:45.640402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:45.640810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:45.642274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 24/53] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:45 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:45.646554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 30/54] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:45.647760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.647891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.718991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:45.722785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:45.753750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a5d8e-a71c-42f1-9ccc-7fe11881b4bb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.756588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285083c4-6d8c-44e8-beb7-0cc2d4f84934 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.793222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.794035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.794324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.794730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.794962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981406ef-508f-4538-b663-f55e8fc3a75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.795323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981406ef-508f-4538-b663-f55e8fc3a75a None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.795619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981406ef-508f-4538-b663-f55e8fc3a75a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.795903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981406ef-508f-4538-b663-f55e8fc3a75a None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.796015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:45.796660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981406ef-508f-4538-b663-f55e8fc3a75a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:45.797395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:45.798102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981406ef-508f-4538-b663-f55e8fc3a75a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:45.799040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 31/55] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:45.804900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:45.826820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 25/56] 10.222.0.30 () {60 vars in 1140 bytes} [Thu Jun 16 18:13:45 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:45.928682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82e9b-d47e-4c25-9a0d-7ebbae7b081a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:45.973336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:45.973979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.974222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.974500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.975387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.domain.id=default, user.id=bd8fd8fbb8b9467bac357373688c6910) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:45.975707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:45.977312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb87e6-91a7-45f4-b90f-adef2a18d800 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:45.988253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 32/57] 10.222.0.30 () {56 vars in 1322 bytes} [Thu Jun 16 18:13:45 2022] GET /identity/v3/role_assignments?user.id=bd8fd8fbb8b9467bac357373688c6910&scope.domain.id=default&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 530 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:47.983972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858420fa-b25a-43ba-b172-c3cc1c278748 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:47.984538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858420fa-b25a-43ba-b172-c3cc1c278748 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:47.984624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858420fa-b25a-43ba-b172-c3cc1c278748 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:47.985946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 26/58] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:47.991191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:47.991631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:47.991765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:47.998604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:48.009527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:48.011301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:48.082691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 33/59] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:48.107009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858420fa-b25a-43ba-b172-c3cc1c278748 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:48.217697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858420fa-b25a-43ba-b172-c3cc1c278748 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:48.238954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:48.254268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b53a2d-b463-4930-9c12-e4c4b8815de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:48.255081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53a2d-b463-4930-9c12-e4c4b8815de0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:48.255369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53a2d-b463-4930-9c12-e4c4b8815de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:48.255705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53a2d-b463-4930-9c12-e4c4b8815de0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:48.256888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b53a2d-b463-4930-9c12-e4c4b8815de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:48.258339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b53a2d-b463-4930-9c12-e4c4b8815de0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:48.260593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 27/60] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:48.266240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:48.366360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aa5db-51c6-42da-aca2-8e9c1a1d9800 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:48.401896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dedc0f-724c-4b20-bcbd-7d3c467a4ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:48.402584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dedc0f-724c-4b20-bcbd-7d3c467a4ffe None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:48.402872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dedc0f-724c-4b20-bcbd-7d3c467a4ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:48.403159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dedc0f-724c-4b20-bcbd-7d3c467a4ffe None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:48.404174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dedc0f-724c-4b20-bcbd-7d3c467a4ffe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:48.406642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dedc0f-724c-4b20-bcbd-7d3c467a4ffe None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:48.415792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:48.433421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 34/61] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:13:48 2022] POST /identity/v3/users => generated 275 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:51.745575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47490b-3e63-4ef6-b883-ff6e65a772e2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:51.746785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47490b-3e63-4ef6-b883-ff6e65a772e2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:51.747319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47490b-3e63-4ef6-b883-ff6e65a772e2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:51.749031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 28/62] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:51.754984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4a43f-ae30-45ec-b52b-3665df42dbdf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:51.755879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4a43f-ae30-45ec-b52b-3665df42dbdf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:51.756389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4a43f-ae30-45ec-b52b-3665df42dbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:51.775222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-68b4a43f-ae30-45ec-b52b-3665df42dbdf None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:51.777476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-68b4a43f-ae30-45ec-b52b-3665df42dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:51.853216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 35/63] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:51.897331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47490b-3e63-4ef6-b883-ff6e65a772e2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.053861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47490b-3e63-4ef6-b883-ff6e65a772e2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.058703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea844-1e39-41fe-a103-b7e2f8a0df19 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.058703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea844-1e39-41fe-a103-b7e2f8a0df19 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.058703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea844-1e39-41fe-a103-b7e2f8a0df19 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.058703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 36/64] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:13:52.064946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd415e1-645d-4acc-af35-1ef35c6e2f73 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.065528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd415e1-645d-4acc-af35-1ef35c6e2f73 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.066034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd415e1-645d-4acc-af35-1ef35c6e2f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.086736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1cd415e1-645d-4acc-af35-1ef35c6e2f73 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:52.089477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1cd415e1-645d-4acc-af35-1ef35c6e2f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:52.120929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.121623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.121913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.122193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.135450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:52.143498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:52.145684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:13:52.147026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 29/65] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:52.151604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.215692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 37/66] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:52.217379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3ca8-3fb6-41be-90d8-3bfc09316715 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.217833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3ca8-3fb6-41be-90d8-3bfc09316715 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.218254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3ca8-3fb6-41be-90d8-3bfc09316715 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.219829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 38/67] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:52.225226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.225731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.226182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.244021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:52.256270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:52.257826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:52.316809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429723c-8fec-437d-a3cc-685bbce40b57 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.323865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:52.358455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 39/68] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:52.360503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.374936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.375977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.376541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.376993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.378289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:52.378782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:52.382779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:52.404140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 30/69] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:52.405939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.506545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca96ef-18db-477c-9639-fcce41d98459 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.552614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc81acb-5a69-4ff7-bb8a-8ac50cd0577f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.555184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.556011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.556337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.556622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.566140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:52.569020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:52.577410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:52.578541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:52.579403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 40/70] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:52.580804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.596318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f09f24-1c10-4c53-b3f3-85c2ff3c9c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.597143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f09f24-1c10-4c53-b3f3-85c2ff3c9c5a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.597431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f09f24-1c10-4c53-b3f3-85c2ff3c9c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.597713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f09f24-1c10-4c53-b3f3-85c2ff3c9c5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.609071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f09f24-1c10-4c53-b3f3-85c2ff3c9c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f1f09f24-1c10-4c53-b3f3-85c2ff3c9c5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:52.617226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:52.618239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:13:52.619216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 31/71] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:52.620023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bbf6c-6178-4171-b3d5-6c172f2b92c0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.620023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bbf6c-6178-4171-b3d5-6c172f2b92c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.620433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bbf6c-6178-4171-b3d5-6c172f2b92c0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.622037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 32/72] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:52.622763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bbf6c-6178-4171-b3d5-6c172f2b92c0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.640086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:52.708283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e98495-deb8-40ff-bf30-0b6a190e6cf0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.746315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bbf6c-6178-4171-b3d5-6c172f2b92c0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.748640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c296a-66b9-44b4-a8ae-800f9c5351f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.749559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c296a-66b9-44b4-a8ae-800f9c5351f7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.749845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c296a-66b9-44b4-a8ae-800f9c5351f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.750127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c296a-66b9-44b4-a8ae-800f9c5351f7 None admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.751422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c296a-66b9-44b4-a8ae-800f9c5351f7 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:52.752960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c296a-66b9-44b4-a8ae-800f9c5351f7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:52.754845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 41/73] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:52.756153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e106f7-09d7-4def-86b9-ee82779bf312 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.756705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e106f7-09d7-4def-86b9-ee82779bf312 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.756705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e106f7-09d7-4def-86b9-ee82779bf312 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.758448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 42/74] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:52.759296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e106f7-09d7-4def-86b9-ee82779bf312 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.789191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.789877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.790172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.790455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.791295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:52.791619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:52.793113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c011c-68ac-49f0-87ea-32cb8f40c5a6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:52.802218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 33/75] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:52.803524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.803823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.804136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.823841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:52.825011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:52.860138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:52.878543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e106f7-09d7-4def-86b9-ee82779bf312 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.888562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:52.904436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 34/76] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:52.905437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.919795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.920518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.920810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.921095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.921862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:52.922181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:52.923587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61042f68-94e6-43f5-b86b-f4f23f1230fc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:52.936758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 43/77] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:52.938340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.938714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.939074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.958049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:52.959285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:53.026597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de1705-5cdc-4cac-b0da-96e822349319 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.042321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 44/78] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:53.043187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.072929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.073705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.074038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.074267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.110000 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.152461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.153548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.154257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.158678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c6074-25d2-40ec-8fd2-4476e263fba1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.200399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.201311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.201587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.201876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.219634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.221162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.222921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 45/79] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:53.223715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.281154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 35/80] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:13:53.281943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.338067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f9872-c193-4812-9781-a87d4cee5e6d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.347893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.380066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.380904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.381195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.381485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.395753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820875c-45e5-44ec-be48-2fba13d4ceed None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.396160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:53.404741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:53.405697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:53.406471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 46/81] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:53.407089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.426911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.436238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.437139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.437429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.437712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.438948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.440459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.442117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 36/82] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:53.442893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.523640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707450e-0823-4aac-af3c-b5f4bf07e34a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.562018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.562612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.562905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.563187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.564431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.564739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75973041-dd75-4122-9ee1-75793625e3bf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.565909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.567574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 47/83] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:53.568370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.599692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.607475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.608176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.608467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.608748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.656850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.673545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b1421-e0d9-4662-9020-02dee49e2b1e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.680257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.682388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.708681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.709562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.709983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.710371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.711913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.713533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.715273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 48/84] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:53.716065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.792259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 37/85] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:13:53.793224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.825714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.831096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d844f57-449d-4b02-bbae-61a8c1ce563a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.867448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.868062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.868369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.868659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.869606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:53.870011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:53.871431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:53.881671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 49/86] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:53.882589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:53.904730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:53.905152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc4073-fb53-4094-b149-076cef61dab6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:53.942500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:53.943175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:53.943467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:53.943752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:53.998789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97692-072c-44af-a68b-4252fe696505 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.030815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.032450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.044595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.045283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.045568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.045846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.046580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.048041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.064577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:54.068361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 50/87] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:13:53 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:54.069194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:54.155362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 38/88] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:13:54.156196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:54.163637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:54.200960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62987906-4446-451d-afb9-9177378e30d2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.248074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.249010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.249435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.249847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.251273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=bd8fd8fbb8b9467bac357373688c6910) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:54.251759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.254097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.265775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 51/89] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/role_assignments?user.id=bd8fd8fbb8b9467bac357373688c6910&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:54.267272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:54.287369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471c13-e608-4922-a3cf-c965d5b76acd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.330042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:54.339103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.340918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.341206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.341488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.343052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.344743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.352042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 39/90] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:54.353178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:54.428190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fcded-bdb8-4197-80dd-ea11b058632b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.511821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.514058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.514565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.515040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.524073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.524284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85e7cc-c402-4cad-ad59-09849b55584c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.524370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.525185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 52/91] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:54.526708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:54.534193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:54.581202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.582632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.583091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.583517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.585833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.589089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.592195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 40/92] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:54.593596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:54.747800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:54.790547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9511-95eb-4c59-8ebe-908706745bb6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.839413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45754f-aa21-4d77-ac42-53820cddb990 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:54.880892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.881602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.881893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.882179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.883065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=8593faba1d9d4e718339612ca5bec325) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:54.883390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:54.889574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:54.894239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:54.895542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:54.896393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:54.897186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:54.909784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 53/93] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/role_assignments?user.id=8593faba1d9d4e718339612ca5bec325&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 259 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:54.910672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:55.006976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:55.008754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:55.096871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:55.125050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:55.174375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8716113-c7a3-4f69-ab21-68f80eae9335 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:55.209358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 41/94] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:13:55.215863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:55.232339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:55.233326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:55.233758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:55.236270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:55.237537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] RBAC: Adding query filter params (role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=8593faba1d9d4e718339612ca5bec325) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:55.238378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:55.240375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:55.257758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 54/95] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:13:54 2022] GET /identity/v3/role_assignments?user.id=8593faba1d9d4e718339612ca5bec325&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:55.365613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496086d-3c12-4b88-aed4-7a6bf30537a4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:55.420737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fedec-ca74-45c6-9eb2-c05cf7933076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:55.422015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fedec-ca74-45c6-9eb2-c05cf7933076 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:55.422476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fedec-ca74-45c6-9eb2-c05cf7933076 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:55.422910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fedec-ca74-45c6-9eb2-c05cf7933076 None admin] PATH_INFO: `/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:55.424575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fedec-ca74-45c6-9eb2-c05cf7933076 None admin] RBAC: Authorizing `identity:get_project(project_id=e38e21b39cf44acf84866ef9f8ba2b0e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:55.426224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fedec-ca74-45c6-9eb2-c05cf7933076 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:55.428347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 42/96] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:13:55 2022] GET /identity/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:55.436177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:55.661034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea22c0-965b-4f42-a293-6abf9071a7f9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:55.718605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:55.731291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:55.732450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:55.732951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:55.733430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:55.735030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=e38e21b39cf44acf84866ef9f8ba2b0e, user.id=8593faba1d9d4e718339612ca5bec325) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:55.735579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:55.738333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472c768-b913-4692-a9c7-af5f9c100406 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:55.751729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 55/97] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:13:55 2022] GET /identity/v3/role_assignments?user.id=8593faba1d9d4e718339612ca5bec325&scope.project.id=e38e21b39cf44acf84866ef9f8ba2b0e&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:57.897363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72525f0b-cec5-4ab2-8334-0818bf470049 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:57.898401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72525f0b-cec5-4ab2-8334-0818bf470049 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:57.898892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72525f0b-cec5-4ab2-8334-0818bf470049 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:57.900575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 43/98] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:57.907048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d23e48-b723-47bf-8360-5ea89e35ca39 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:57.908227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d23e48-b723-47bf-8360-5ea89e35ca39 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:57.908992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d23e48-b723-47bf-8360-5ea89e35ca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:57.939172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f7d23e48-b723-47bf-8360-5ea89e35ca39 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:57.940850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f7d23e48-b723-47bf-8360-5ea89e35ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:58.035197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 56/99] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:58.054931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9eb3a-1ebe-428c-a249-50215722beb6 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.055756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9eb3a-1ebe-428c-a249-50215722beb6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.056459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9eb3a-1ebe-428c-a249-50215722beb6 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.059205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 44/100] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:13:58.065295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.065896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.066396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.069468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9eb3a-1ebe-428c-a249-50215722beb6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:58.091236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:58.092869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:58.133980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:58.225851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 57/101] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:58.261713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:58.272586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9eb3a-1ebe-428c-a249-50215722beb6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:58.358469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:58.359450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.359857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.360234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.389187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:58.397931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:58.398905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:13:58.399725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:13:58.404564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 45/102] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:13:58.411957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:58.435106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:58.562686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dac80-cd30-4037-b723-ff9d726e5e50 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:58.599895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:58.611107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126affc-5a37-49a1-a7d9-947412b13dcc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:58.622942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197742-ba2c-4a89-b009-c673bc5b725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:58.624804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197742-ba2c-4a89-b009-c673bc5b725f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.625631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197742-ba2c-4a89-b009-c673bc5b725f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.626391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197742-ba2c-4a89-b009-c673bc5b725f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.647020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197742-ba2c-4a89-b009-c673bc5b725f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-84197742-ba2c-4a89-b009-c673bc5b725f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:58.667998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:58.669845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:13:58.671373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:58.676307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 58/103] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:13:58.678461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126067d-836b-4921-a9e4-bd17cc6caed0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.679319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126067d-836b-4921-a9e4-bd17cc6caed0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.679914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126067d-836b-4921-a9e4-bd17cc6caed0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.682963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 59/104] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:58.684329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126067d-836b-4921-a9e4-bd17cc6caed0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:58.709541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:58.710421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.710624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.710864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.711692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:58.712037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:58.714137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376856e-498a-43cb-8be6-20e7b9b2f28e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:58.733217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 46/105] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:13:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:58.734635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82acf64-3403-4844-8de6-4ef79296f332 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.734968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82acf64-3403-4844-8de6-4ef79296f332 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.735280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82acf64-3403-4844-8de6-4ef79296f332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.754746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c82acf64-3403-4844-8de6-4ef79296f332 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:58.756096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c82acf64-3403-4844-8de6-4ef79296f332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:58.837406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 47/106] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:58.839321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60bf57-1d95-4d7b-b4ab-a46a54afe57f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.839624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60bf57-1d95-4d7b-b4ab-a46a54afe57f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.839974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60bf57-1d95-4d7b-b4ab-a46a54afe57f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.841747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 48/107] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:58.842120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60bf57-1d95-4d7b-b4ab-a46a54afe57f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:58.855093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126067d-836b-4921-a9e4-bd17cc6caed0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:58.884513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:58.915115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:58.916895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.917382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.917869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.918849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:58.919369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:58.921003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a738c0-9fbf-42a2-9b13-f331a208d5d1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:58.931801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 60/108] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:13:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:58.933840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:58.934354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:58.934821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:58.957567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:58.959180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:58.987956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.017512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60bf57-1d95-4d7b-b4ab-a46a54afe57f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.067917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 61/109] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:59.069459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.078079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.079052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.079342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.079616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] PATH_INFO: `/v3/users/b61a45d09c144d13b925e5f9a593fb01` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.113822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=b61a45d09c144d13b925e5f9a593fb01)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.116566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:59.118816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 49/110] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:58 2022] GET /identity/v3/users/b61a45d09c144d13b925e5f9a593fb01 => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:59.120296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.227305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c368c-abf5-46e5-ba00-e42a8f9e4938 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.242488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.252321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89cf84-0b1d-4061-8fe8-4538500c0fa9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.270338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc6a9-d1e9-499c-b3c8-66e9f69ec2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.271066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc6a9-d1e9-499c-b3c8-66e9f69ec2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.271353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc6a9-d1e9-499c-b3c8-66e9f69ec2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.271629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc6a9-d1e9-499c-b3c8-66e9f69ec2d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.272911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc6a9-d1e9-499c-b3c8-66e9f69ec2d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.274564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc6a9-d1e9-499c-b3c8-66e9f69ec2d6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:59.277220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 62/111] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:59.278542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914003b-15fa-4e34-8e20-56386aac0ce2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.279091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914003b-15fa-4e34-8e20-56386aac0ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.279374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914003b-15fa-4e34-8e20-56386aac0ce2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.280677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 63/112] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:59.283201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914003b-15fa-4e34-8e20-56386aac0ce2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.310250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.311142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.311423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.311726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.313015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.314546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:59.316298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 50/113] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:13:59 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:59.317541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.342043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.423178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914003b-15fa-4e34-8e20-56386aac0ce2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.446713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5173d-57be-4add-9b76-495d9cf744a5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.462977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ca37-b2b6-4e70-b665-410fd4122889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.464326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ca37-b2b6-4e70-b665-410fd4122889 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.464777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ca37-b2b6-4e70-b665-410fd4122889 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.465190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ca37-b2b6-4e70-b665-410fd4122889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.477089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56ca37-b2b6-4e70-b665-410fd4122889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-5e56ca37-b2b6-4e70-b665-410fd4122889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:13:59.485608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:13:59.486721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:13:59.487523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 64/114] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:13:59.488601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28115f6f-edad-49af-aa76-617c3adcfdca None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.488601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28115f6f-edad-49af-aa76-617c3adcfdca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.488770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28115f6f-edad-49af-aa76-617c3adcfdca None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.490287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 65/115] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:13:59.490914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28115f6f-edad-49af-aa76-617c3adcfdca None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.491364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568619e-f35a-4762-864c-56b928281e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.494011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568619e-f35a-4762-864c-56b928281e58 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.494011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568619e-f35a-4762-864c-56b928281e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.494011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568619e-f35a-4762-864c-56b928281e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.538786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.575564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568619e-f35a-4762-864c-56b928281e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.577113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568619e-f35a-4762-864c-56b928281e58 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:59.592984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.620628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28115f6f-edad-49af-aa76-617c3adcfdca None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.659469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.660180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.660467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.660746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.661479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.662987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:59.692498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 51/116] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:13:59.694070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 66/117] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:13:59 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:59.694184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.694403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.694659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.705161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.719651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:59.720956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:59.813629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 52/118] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:13:59.813629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:59.826862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.846794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d005f-809f-4def-9c30-bfaaca22a119 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.864358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:59.893429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578a137-4afd-49bc-a405-cb8e9c01a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.893666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578a137-4afd-49bc-a405-cb8e9c01a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.893884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578a137-4afd-49bc-a405-cb8e9c01a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.894207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578a137-4afd-49bc-a405-cb8e9c01a4e5 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.895725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578a137-4afd-49bc-a405-cb8e9c01a4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.896935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578a137-4afd-49bc-a405-cb8e9c01a4e5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:59.899596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 67/119] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:13:59 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:59.901187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.901736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.902172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.921189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:59.922515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:59.938032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707a13-c0c9-4853-8f93-bed803c98232 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:59.978416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:59.979930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:59.980335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:59.980621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:59.981442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:59.981858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:59.984461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.005880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 53/120] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:00.005880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.008443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 68/121] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:00.012574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.114920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:00.131592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733652f-38b8-4172-b677-a9ddf18ff7d7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.134842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:00.137614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070d8e1-2156-4a50-a982-3762e085251c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.174820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.175707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.175997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.176299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.177544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:00.179004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.180759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 54/122] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:00.181519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.188729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.189581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.189882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.190161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.202734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:00.210621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:00.211607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 69/123] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:00.218280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.304728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0972f58-0ad6-4ffa-b8fd-947839fcc258 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.335435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697545dc-b888-43bd-b057-32d6a08b270e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.342247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.343347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.343641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.343925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/8593faba1d9d4e718339612ca5bec325/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.346011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] RBAC: Authorizing `identity:create_grant(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user_id=8593faba1d9d4e718339612ca5bec325, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:00.347488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.372176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 55/124] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:00 2022] PUT /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/8593faba1d9d4e718339612ca5bec325/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:00.372176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.389160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.390809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.391410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.391707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.393856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:00.396478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.399804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 70/125] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:00.402344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.433170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:00.455196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:00.569858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f778e-3bb6-4b94-a8aa-1dbf7c5c411e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.570414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef6df-fe90-4c21-a5fd-e699d5705b40 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.615333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.616689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.617301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.617782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.618013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.618611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.618900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.619704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.620218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] RBAC: Adding query filter params (role.id=a1a7ce3d503044739a4c81a7fba52623, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=b61a45d09c144d13b925e5f9a593fb01) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:00.620840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:00.622825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.629695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 71/126] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/role_assignments?user.id=b61a45d09c144d13b925e5f9a593fb01&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=a1a7ce3d503044739a4c81a7fba52623 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:00.630902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.636139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:00.644502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:00.645459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:00.646242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 56/127] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:00.647377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.795458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:00.805567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603e34-d5cb-43c6-a4e3-2193b70eefa3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.832500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:00.858834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.860076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.860517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.860941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.862578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:00.864350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.866534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 57/128] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:00.867621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:00.914570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6c557-8c7d-440b-99b6-6d07e8adb3e9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:00.965835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:00.966678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:00.967226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:00.967611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:00.968647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:00.969044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:00.970626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:00.988025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 72/129] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:00.988749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.004969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ca6f-6c3f-4b8a-92cb-42e34fcbc84c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.049197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.050796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.051436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.052023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.053052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:01.053578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.055255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.076360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 58/130] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:01.084082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.131193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af925ff-b225-4367-b4bd-f658b6abb604 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.163110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:01.199863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.200735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.201060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.201440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/bd8fd8fbb8b9467bac357373688c6910/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.203670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user_id=bd8fd8fbb8b9467bac357373688c6910, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.205423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.226119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 73/131] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:00 2022] PUT /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/bd8fd8fbb8b9467bac357373688c6910/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:01.226298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:01.232132 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.369908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201ae61-ff65-4a3c-b82d-3abac96bb35c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.408975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2216778-5a20-475f-be6f-e210b4c464e6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.448392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.449241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.449528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.449806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.454483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.455908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.456709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.457843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 59/132] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:01 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:01.457970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.458122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.458474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.464739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.466358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.468242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 74/133] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:01 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:01.470278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.474037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.615099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:01.683952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74012dd-64b8-4ba4-a2f3-25616ee80b08 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.701199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:01.720869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e4297-b44d-4411-9a08-5a7f4a14edc1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.754558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.755439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.755722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.755998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.758233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.759716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.762434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 75/134] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:01 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:01.768287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.788811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.790100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.790681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.791287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] PATH_INFO: `/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.793633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=e38e21b39cf44acf84866ef9f8ba2b0e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.795418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.797565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 60/135] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:01 2022] GET /identity/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e => generated 312 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:01.803716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:01.888427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1952f28-98e5-4b2e-984b-9b2f432d3240 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:01.921881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1553d6-9e32-4208-92ff-85470e95fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:01.922828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1553d6-9e32-4208-92ff-85470e95fdc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:01.923108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1553d6-9e32-4208-92ff-85470e95fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:01.923383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1553d6-9e32-4208-92ff-85470e95fdc5 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/8593faba1d9d4e718339612ca5bec325/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:01.925975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1553d6-9e32-4208-92ff-85470e95fdc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user_id=8593faba1d9d4e718339612ca5bec325, role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:01.927920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1553d6-9e32-4208-92ff-85470e95fdc5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:01.933162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:01.945388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 76/136] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:01 2022] PUT /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/8593faba1d9d4e718339612ca5bec325/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:01.983177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15199101-bcb9-4943-aa4c-81e3c16dc5c1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:02.030037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd8256-9feb-4fa9-87bc-6b774ec8460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:02.030965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd8256-9feb-4fa9-87bc-6b774ec8460e None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:02.031260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd8256-9feb-4fa9-87bc-6b774ec8460e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:02.031546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd8256-9feb-4fa9-87bc-6b774ec8460e None admin] PATH_INFO: `/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e/users/8593faba1d9d4e718339612ca5bec325/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:02.033739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd8256-9feb-4fa9-87bc-6b774ec8460e None admin] RBAC: Authorizing `identity:create_grant(project_id=e38e21b39cf44acf84866ef9f8ba2b0e, user_id=8593faba1d9d4e718339612ca5bec325, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:02.035227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd8256-9feb-4fa9-87bc-6b774ec8460e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:02.058896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 61/137] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:01 2022] PUT /identity/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e/users/8593faba1d9d4e718339612ca5bec325/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:14:02.425926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a8e0e-8800-4ec5-9881-3a915baca3c2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:02.427029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a8e0e-8800-4ec5-9881-3a915baca3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:02.427514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a8e0e-8800-4ec5-9881-3a915baca3c2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:02.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 77/138] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:02.436264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:02.436566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:02.436860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:02.461309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:02.462648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:02.483528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:02.589466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 62/139] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:02.615087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a8e0e-8800-4ec5-9881-3a915baca3c2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:02.785885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a8e0e-8800-4ec5-9881-3a915baca3c2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:02.833665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68ce1c-dd1b-451e-b50d-f3ebe2f8aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:02.834575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68ce1c-dd1b-451e-b50d-f3ebe2f8aa9a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:02.835028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68ce1c-dd1b-451e-b50d-f3ebe2f8aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:02.835448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68ce1c-dd1b-451e-b50d-f3ebe2f8aa9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:02.838401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68ce1c-dd1b-451e-b50d-f3ebe2f8aa9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:02.839687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68ce1c-dd1b-451e-b50d-f3ebe2f8aa9a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:02.841647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 78/140] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:02.848540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:02.976440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592859d-a89b-4085-89e8-2882ed586548 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:03.018616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720fd3d-1294-4618-9c32-cd6100abfb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:03.019400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720fd3d-1294-4618-9c32-cd6100abfb13 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:03.019781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720fd3d-1294-4618-9c32-cd6100abfb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:03.020082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720fd3d-1294-4618-9c32-cd6100abfb13 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:03.021158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720fd3d-1294-4618-9c32-cd6100abfb13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:03.022654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720fd3d-1294-4618-9c32-cd6100abfb13 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:03.057586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 63/141] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:02 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:14:03.975712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ed89-e0ea-40c5-89e2-ca9d70e4e0db None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:03.976444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ed89-e0ea-40c5-89e2-ca9d70e4e0db None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:03.976444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9ed89-e0ea-40c5-89e2-ca9d70e4e0db None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:03.977710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 79/142] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:03.985232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:03.985844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:03.985844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:03.992943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:04.006756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:04.008025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:04.092279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 64/143] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:04.116871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ed89-e0ea-40c5-89e2-ca9d70e4e0db None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:04.212217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:04.264007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9ed89-e0ea-40c5-89e2-ca9d70e4e0db None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:04.309079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe6e26-a8df-424e-967d-9f53d34a69e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:04.309926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe6e26-a8df-424e-967d-9f53d34a69e1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:04.310303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe6e26-a8df-424e-967d-9f53d34a69e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:04.310550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe6e26-a8df-424e-967d-9f53d34a69e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:04.325540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe6e26-a8df-424e-967d-9f53d34a69e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-31fe6e26-a8df-424e-967d-9f53d34a69e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:04.333971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:04.334868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:04.336064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 80/144] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:04.342398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:04.402740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472d18-28df-4511-9173-ca7c42c50446 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:04.403111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472d18-28df-4511-9173-ca7c42c50446 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:04.403415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472d18-28df-4511-9173-ca7c42c50446 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:04.405045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 81/145] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:04.420164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:04.420457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:04.420733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:04.449636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:04.450887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:04.546441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098de52-61fc-468e-919c-f542a69c1fa9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:04.613807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 82/146] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:04.640285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:04.641026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:04.641326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:04.641621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:04.642432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:04.642758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:04.644268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:04.645650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:04.646314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:04.658326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 65/147] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:04.677650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:04.777273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:04.859367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf44e7f-9142-4762-9f33-8e548b1ca24b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:04.918560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db1d3f-91cc-4c3d-9adc-41283d17c090 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:04.937633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:04.938863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:04.939260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:04.939717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:04.966836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:04.983250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:04.984682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:04.985443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:04.988763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 83/148] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:04.996845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:05.009238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:05.010862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:05.011151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:05.011581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:05.013745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:05.015911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:05.019501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 66/149] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:04 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:05.025441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:05.180599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:05.203400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f77599-9d9b-48b4-a4bd-2accbf2bd484 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:05.231572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4d09-f74a-42ed-9445-e8b0bb7b7167 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:05.241238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:05.282021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:05.282964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:05.283393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:05.283813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:05.284817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:05.285308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:05.286962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:05.310368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:05.310693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 84/150] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:05.311434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:05.311791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:05.312107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:05.316030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:05.433909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:05.435402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:05.561270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c06cea-8349-4c1c-8fd0-b1a0c70668ce None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:05.573668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:05.639952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 67/151] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:05.643748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:05.644697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:05.644983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:05.645266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] PATH_INFO: `/v3/users/b61a45d09c144d13b925e5f9a593fb01` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:05.645794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:05.646547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] RBAC: Authorizing `identity:get_user(user_id=b61a45d09c144d13b925e5f9a593fb01)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:05.648023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:05.650084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 85/152] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:05 2022] GET /identity/v3/users/b61a45d09c144d13b925e5f9a593fb01 => generated 289 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:05.657148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:05.783124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:05.864017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d0eb9-d4b2-4d5a-8b20-7a1ad4e9e67c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:05.920037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38cea-cbee-41f9-ac89-ff9116c9dd11 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:05.930803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871787c-f3a6-4ce5-ad78-b44e8bbf820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:05.932305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871787c-f3a6-4ce5-ad78-b44e8bbf820d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:05.932956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871787c-f3a6-4ce5-ad78-b44e8bbf820d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:05.932956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871787c-f3a6-4ce5-ad78-b44e8bbf820d None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:05.934086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871787c-f3a6-4ce5-ad78-b44e8bbf820d None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:05.935558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871787c-f3a6-4ce5-ad78-b44e8bbf820d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:05.944590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 68/153] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:05 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:05.946359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea46fb5-e2e5-4c6a-80e6-50d31463acd4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:05.946652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea46fb5-e2e5-4c6a-80e6-50d31463acd4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:05.946929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea46fb5-e2e5-4c6a-80e6-50d31463acd4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:05.948466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 69/154] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:05.950989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea46fb5-e2e5-4c6a-80e6-50d31463acd4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:06.005743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b0fb5-23fc-4f8c-b4d8-eec6710071d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:06.006642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b0fb5-23fc-4f8c-b4d8-eec6710071d9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.007059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b0fb5-23fc-4f8c-b4d8-eec6710071d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.007344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b0fb5-23fc-4f8c-b4d8-eec6710071d9 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.008434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:06.008726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b0fb5-23fc-4f8c-b4d8-eec6710071d9 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:06.016420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b0fb5-23fc-4f8c-b4d8-eec6710071d9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:06.018611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 86/155] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:05 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:06.019843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.020156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.020439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.056233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:06.058146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:06.133709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea46fb5-e2e5-4c6a-80e6-50d31463acd4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:06.187574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:06.188287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.188568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.188844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.189716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=8593faba1d9d4e718339612ca5bec325) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:06.190032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:06.191527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:06.198452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 70/156] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:05 2022] GET /identity/v3/role_assignments?user.id=8593faba1d9d4e718339612ca5bec325&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:06.205503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:06.262717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 87/157] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:06.312039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:06.356452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:06.362680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22861fb-a88b-4421-8f8d-d5fea07bffe4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:06.406661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:06.417779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:06.418711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.418993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.419266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/b61a45d09c144d13b925e5f9a593fb01/roles/a1a7ce3d503044739a4c81a7fba52623` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.422771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] RBAC: Authorizing `identity:create_grant(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user_id=b61a45d09c144d13b925e5f9a593fb01, role_id=a1a7ce3d503044739a4c81a7fba52623)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:06.424277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:06.450367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 71/158] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:06 2022] PUT /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/users/b61a45d09c144d13b925e5f9a593fb01/roles/a1a7ce3d503044739a4c81a7fba52623 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:06.503868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d785c-0b42-4ef2-bccc-6071c832f419 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:06.560780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b1d13-d753-477e-9469-019b57597a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:06.561617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b1d13-d753-477e-9469-019b57597a3a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.561901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b1d13-d753-477e-9469-019b57597a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.562236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b1d13-d753-477e-9469-019b57597a3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.578760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b1d13-d753-477e-9469-019b57597a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a05b1d13-d753-477e-9469-019b57597a3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:06.586872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:06.587836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:06.589371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 88/159] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:06.596206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:06.742474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2918c-f51e-4bd7-9e39-6d8dc282f71e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:06.747808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841430d7-b7b6-4bfc-acbe-20069873ea50 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.748128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841430d7-b7b6-4bfc-acbe-20069873ea50 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.748415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841430d7-b7b6-4bfc-acbe-20069873ea50 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.750136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 89/160] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:06.756681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.756990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.757237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.778176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:06.779678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:06.795945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:06.796683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:06.796962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:06.797238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:06.798133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:06.798302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:06.799720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:06.813905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 72/161] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:06.820015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:06.865625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:06.928450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 90/162] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:06.966500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:07.006474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:07.014885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3362c1a-1354-4202-a058-c3c05abe4461 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:07.057867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d0252-d8ce-4f7b-a6d8-22fa6e516889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:07.058915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d0252-d8ce-4f7b-a6d8-22fa6e516889 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:07.059198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d0252-d8ce-4f7b-a6d8-22fa6e516889 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:07.059471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d0252-d8ce-4f7b-a6d8-22fa6e516889 None admin] PATH_INFO: `/v3/users/bd8fd8fbb8b9467bac357373688c6910` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:07.060790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d0252-d8ce-4f7b-a6d8-22fa6e516889 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8fd8fbb8b9467bac357373688c6910)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:07.062247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d0252-d8ce-4f7b-a6d8-22fa6e516889 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:07.064125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 73/163] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:06 2022] GET /identity/v3/users/bd8fd8fbb8b9467bac357373688c6910 => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:07.065296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967270e-6969-42a8-b050-4e2b035cace5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:07.065589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967270e-6969-42a8-b050-4e2b035cace5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:07.065868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967270e-6969-42a8-b050-4e2b035cace5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:07.067127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 74/164] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:07.070490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967270e-6969-42a8-b050-4e2b035cace5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:07.108358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f31d28-b17e-49c7-aa55-011d8df31110 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:07.193483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fadf3f-71d7-4d82-8d38-9463981d44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:07.194334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fadf3f-71d7-4d82-8d38-9463981d44ae None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:07.194672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fadf3f-71d7-4d82-8d38-9463981d44ae None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:07.195140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fadf3f-71d7-4d82-8d38-9463981d44ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:07.215496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fadf3f-71d7-4d82-8d38-9463981d44ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-63fadf3f-71d7-4d82-8d38-9463981d44ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:07.231126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:07.233284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:07.235149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 91/165] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:07.239750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:07.240032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:07.240336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:07.268446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:07.269740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:07.283972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:07.365357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967270e-6969-42a8-b050-4e2b035cace5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:07.409962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 92/166] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:07.410957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:07.451455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:07.466169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:07.466843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:07.467127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:07.467404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:07.608992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa262c-a61e-4001-b2e0-a238cb60c4ac None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:07.649284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:07.651652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:07.685833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:07.686532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:07.686811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:07.687105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:07.687867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:07.688218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:07.689733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:07.710271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 93/167] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:07.711136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:07.876860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:07.907323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204290-5b0f-4574-9f09-91673e7bb7fb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:07.914338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:07.948006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 75/168] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 877 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:07.948803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.004910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee73b9-c4c3-4dee-b530-d89aac9eea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.005749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee73b9-c4c3-4dee-b530-d89aac9eea6e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.006034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee73b9-c4c3-4dee-b530-d89aac9eea6e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.006313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee73b9-c4c3-4dee-b530-d89aac9eea6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.040120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee73b9-c4c3-4dee-b530-d89aac9eea6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-10ee73b9-c4c3-4dee-b530-d89aac9eea6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:08.046423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:08.047387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:08.048623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 94/169] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:08.054122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d91cde-5f41-4275-9c9d-b383ce75e945 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.054436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d91cde-5f41-4275-9c9d-b383ce75e945 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.054673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d91cde-5f41-4275-9c9d-b383ce75e945 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.056157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 95/170] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:08.057645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d91cde-5f41-4275-9c9d-b383ce75e945 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.125621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531c70-2408-47ee-a4c6-afb0a8ec9657 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:08.194987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.195864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.196187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.196458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.199707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:08.203120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:08.206737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 76/171] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:07 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:08.207558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.262480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d91cde-5f41-4275-9c9d-b383ce75e945 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:08.300122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e294871-53e4-4b8f-bd62-aede352955ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.301001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e294871-53e4-4b8f-bd62-aede352955ab None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.301279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e294871-53e4-4b8f-bd62-aede352955ab None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.301553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e294871-53e4-4b8f-bd62-aede352955ab None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.302815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e294871-53e4-4b8f-bd62-aede352955ab None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:08.304346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e294871-53e4-4b8f-bd62-aede352955ab None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:08.306054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 96/172] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:08.307577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.312469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.346969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:08.348199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:08.356428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:08.364433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:08.430709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 97/173] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:08.431393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.469314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12820ff5-740f-4edd-84a3-1cde9b10bf07 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:08.524870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.526751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.527043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.527385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.528183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:08.528502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:08.530425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:08.540173 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 77/174] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:08.541066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.620290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f5b1d-0f32-4895-8ecf-7b7712285ac8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:08.659913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.660985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.661481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.661956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.676593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:08.688230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccf21-f036-43fd-ac00-dbf9a6d0eb9a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:08.705593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:08.727288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.727878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.728203 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.728516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.730132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=bd8fd8fbb8b9467bac357373688c6910) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:08.730419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:08.731850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:08.744107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 78/175] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/role_assignments?user.id=bd8fd8fbb8b9467bac357373688c6910&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:08.744107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.744450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:08.744450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:08.869582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 98/176] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:08.870745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:08.881588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522204dd-30fe-4789-bad1-5dbb30c97c7e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:08.927351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:08.936389 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271289-1740-427a-8e50-82a9a466b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:08.937627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271289-1740-427a-8e50-82a9a466b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.939167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271289-1740-427a-8e50-82a9a466b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.939167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271289-1740-427a-8e50-82a9a466b8f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.953252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6271289-1740-427a-8e50-82a9a466b8f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c6271289-1740-427a-8e50-82a9a466b8f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:08.962057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:08.963255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:08.964294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 79/177] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:08.965905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896da9a-b415-4b89-bb39-6bd685ece6e7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.965905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896da9a-b415-4b89-bb39-6bd685ece6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.966077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896da9a-b415-4b89-bb39-6bd685ece6e7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.969986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 80/178] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:08.969986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8c9c-7f02-4d22-ac7f-c2d053dd4daf None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:08.970266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8c9c-7f02-4d22-ac7f-c2d053dd4daf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:08.970543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8c9c-7f02-4d22-ac7f-c2d053dd4daf None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:08.972096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 81/179] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:08.976135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8c9c-7f02-4d22-ac7f-c2d053dd4daf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.007507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2c5d8-00ab-465b-be84-ea4e9620eb45 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.025209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.054911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.055949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.056360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.056864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] PATH_INFO: `/v3/users/8593faba1d9d4e718339612ca5bec325` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.059062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] RBAC: Authorizing `identity:get_user(user_id=8593faba1d9d4e718339612ca5bec325)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.061310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.064352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 99/180] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/users/8593faba1d9d4e718339612ca5bec325 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:09.065279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.111757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8c9c-7f02-4d22-ac7f-c2d053dd4daf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.161214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba19839-017a-4000-ba78-1c913b0459c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.162103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba19839-017a-4000-ba78-1c913b0459c7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.162389 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba19839-017a-4000-ba78-1c913b0459c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.162666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba19839-017a-4000-ba78-1c913b0459c7 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.164429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba19839-017a-4000-ba78-1c913b0459c7 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.165873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba19839-017a-4000-ba78-1c913b0459c7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.173658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 82/181] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:08 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:09.173658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.173658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.173658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.192296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:09.193253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f7db-dc3f-4587-8212-9a3a1454b5ac None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.193512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:09.206625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.235730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.236501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.237007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.237114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.238092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:09.238200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.239585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb545c1-5ade-46e7-8f82-6d9b67bc60ff None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.249666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 100/182] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:09.250771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.251062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.251339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.269461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:09.270567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:09.277134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 83/183] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:09.277990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.321365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.372128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 101/184] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:09.373378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.407602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22cb76-c6e2-4d46-b4e9-18bedc0d1e36 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.444368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.445032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.445318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.445587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.510715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c9d4a-1940-4029-b4d2-77cb7aea73bb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.514481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.532525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.533964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.552841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.553497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.553773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.554048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.554883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] RBAC: Adding query filter params (role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=8593faba1d9d4e718339612ca5bec325) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:09.555192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.556844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.562789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 102/185] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/role_assignments?user.id=8593faba1d9d4e718339612ca5bec325&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:09.563559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.584104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.658814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 84/186] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:09.659743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.691951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d406c-34e0-4532-8a48-36d154ea93c2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.740745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.741636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.741916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.742191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] PATH_INFO: `/v3/users/f748783700994248bec69f5d9e1515ed` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.764018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] RBAC: Authorizing `identity:get_user(user_id=f748783700994248bec69f5d9e1515ed)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.766217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.768354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 103/187] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/users/f748783700994248bec69f5d9e1515ed => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:09.773010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.795227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d083f99-8d14-4dde-b9c5-3ca1fe9d32d2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.803876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.836481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10238cd4-e5c9-47b7-8994-9277c89bff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.837411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238cd4-e5c9-47b7-8994-9277c89bff0f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.837705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238cd4-e5c9-47b7-8994-9277c89bff0f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.838128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10238cd4-e5c9-47b7-8994-9277c89bff0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.845722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:09.850827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10238cd4-e5c9-47b7-8994-9277c89bff0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-10238cd4-e5c9-47b7-8994-9277c89bff0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:09.860570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:09.861465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:09.862532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 85/188] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:09.863466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418012d-03f3-4289-97c6-49bce8831aae None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.863466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418012d-03f3-4289-97c6-49bce8831aae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.863584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418012d-03f3-4289-97c6-49bce8831aae None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.865219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 86/189] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:09.865801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418012d-03f3-4289-97c6-49bce8831aae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.890010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86922f3-e9c5-441c-b07d-385e718a3398 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:09.925749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:09.926552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:09.926841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:09.927114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:09.928315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:09.929757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:09.931386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 104/190] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:09.933611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:09.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418012d-03f3-4289-97c6-49bce8831aae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.007885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.008779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.009067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.009344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] PATH_INFO: `/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.011211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] RBAC: Authorizing `identity:get_project(project_id=e38e21b39cf44acf84866ef9f8ba2b0e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.012722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.014423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 87/191] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/projects/e38e21b39cf44acf84866ef9f8ba2b0e => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:10.015182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.038011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.048362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e8899-7934-4bb3-a2e4-7b7cc2c0a6a9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.086482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.090238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.090940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.091251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.091529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.131161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dcf8a-bd39-4b6c-ab55-a85ec6df15c9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.166000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.167567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.168473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.168473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.168473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.168473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.170366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:10.170678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.172098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d129-e7eb-409c-ad37-e13c22323e87 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.180619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 88/192] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:10.180619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.180619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.180619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.202176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:10.203396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:10.267609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.271516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 89/193] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:10.272906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.282706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 105/194] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:10.282706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.335815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.379728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e266e39-1cdb-4197-9ac2-2e6acd97d686 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.386127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98904373-9808-447f-8203-a54aff2afe64 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.415309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.416533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.416953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.417227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.417931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.419379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.422268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.422927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.423209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.423484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.424348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=e38e21b39cf44acf84866ef9f8ba2b0e, user.id=8593faba1d9d4e718339612ca5bec325) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:10.424662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.426090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.433120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 106/195] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/role_assignments?user.id=8593faba1d9d4e718339612ca5bec325&scope.project.id=e38e21b39cf44acf84866ef9f8ba2b0e&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:10.433120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.444640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 90/196] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:10 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:10.444640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.515452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.576092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fc11f-a14d-4b72-9545-1f9ab0116a1f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.596366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35196-476d-46b7-a1cb-94306313a836 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.623362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.624624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.625019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.625446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] PATH_INFO: `/v3/users/f748783700994248bec69f5d9e1515ed` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.627120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] RBAC: Authorizing `identity:get_user(user_id=f748783700994248bec69f5d9e1515ed)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.627981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.628934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.631711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 107/197] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/users/f748783700994248bec69f5d9e1515ed => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:10.632763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.651149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.652946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.653239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.653679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.655463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.657722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.660208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 91/198] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:10.661205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.774125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4cf27-b0fc-41bb-b1c2-ae878d1021e9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.809041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.810571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5dc3c-0b81-4d64-b684-b0c1798b8ad7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:10.823768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.824700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.826364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.826364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.839894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:10.848108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:10.849040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:10.849814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 108/199] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:10.850891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:10.862421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:10.863098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:10.863369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:10.863645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:10.922464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:10.945010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:10.946697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:10.991879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268c66a-a187-48f3-9ebe-90c34cecbe75 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:11.040490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:11.041978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:11.042266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:11.042545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:11.043684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=f748783700994248bec69f5d9e1515ed) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:11.044440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:11.045928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:11.054393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 109/200] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/role_assignments?user.id=f748783700994248bec69f5d9e1515ed&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:11.055268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:11.091771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 92/201] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:11.097891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:11.117048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:11.188187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbcdd1-1ca9-457f-92a2-27b188ef20f5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:11.234101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:11.234800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:11.235087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:11.235366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:11.236180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:11.236498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:11.237905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:11.254841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 110/202] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:11.262175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:11.337230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:11.375631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742163d-c32d-474a-b330-4a3707308a9a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:11.436109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:11.438651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:11.439188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:11.439783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:11.441714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:11.443473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:11.446452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 93/203] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:11 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:11.455498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:11.480072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046b15f-f391-4b77-ba3c-2d807c2d1826 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:11.508360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:11.560086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:11.561011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:11.561296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:11.561575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] PATH_INFO: `/v3/users/b61a45d09c144d13b925e5f9a593fb01` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:11.566337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] RBAC: Authorizing `identity:get_user(user_id=b61a45d09c144d13b925e5f9a593fb01)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:11.569170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:11.571432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 111/204] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:11 2022] GET /identity/v3/users/b61a45d09c144d13b925e5f9a593fb01 => generated 289 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:11.580866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:11.669475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea3d6e-b387-4ae0-b4e1-ac80a013dccf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:11.734145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeec92a-bca3-471e-b058-8c253a37ff49 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:11.771009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:11.771770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:11.772072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:11.772366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:11.784806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:11.785491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:11.785776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:11.786051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:11.786899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] RBAC: Adding query filter params (role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=f748783700994248bec69f5d9e1515ed) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:11.787214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:11.788716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:11.806122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 94/205] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:11 2022] GET /identity/v3/role_assignments?user.id=f748783700994248bec69f5d9e1515ed&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:11.827061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:11.853583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:11.855542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:11.976716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 112/206] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:11.982694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:12.000076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:12.151688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4724c89-d2f0-401d-9d20-2956529f7a22 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:12.194615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1ae11-0aa3-4ded-9419-d0e648b99486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:12.195493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ae11-0aa3-4ded-9419-d0e648b99486 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:12.195779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ae11-0aa3-4ded-9419-d0e648b99486 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:12.196069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1ae11-0aa3-4ded-9419-d0e648b99486 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:12.197379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1ae11-0aa3-4ded-9419-d0e648b99486 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:12.198835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1ae11-0aa3-4ded-9419-d0e648b99486 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:12.200764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 95/207] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:11 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:12.206708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:12.314685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:12.345190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9cf97-136c-4343-ae74-a24dd26c7533 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:12.398746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:12.399414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:12.399696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:12.399972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:12.401037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] RBAC: Adding query filter params (role.id=a1a7ce3d503044739a4c81a7fba52623, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d, user.id=b61a45d09c144d13b925e5f9a593fb01) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:12.401351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:12.402795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3757e5b-696d-4295-bc59-b5311b74f09a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:12.412944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 113/208] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:12 2022] GET /identity/v3/role_assignments?user.id=b61a45d09c144d13b925e5f9a593fb01&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=a1a7ce3d503044739a4c81a7fba52623 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:13.145378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1025ba-eaff-402d-864f-b763a721c6b5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.145893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1025ba-eaff-402d-864f-b763a721c6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.145969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1025ba-eaff-402d-864f-b763a721c6b5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.147405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 96/209] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:13.153592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ecc3-462a-4afd-b6f2-11e4bd35067a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.153871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ecc3-462a-4afd-b6f2-11e4bd35067a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.154401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ecc3-462a-4afd-b6f2-11e4bd35067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.171952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-09c5ecc3-462a-4afd-b6f2-11e4bd35067a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:13.173506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-09c5ecc3-462a-4afd-b6f2-11e4bd35067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:13.259032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 114/210] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:13.276466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eccc9f-5cdb-42ea-a802-de1bd31780ba None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.276756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eccc9f-5cdb-42ea-a802-de1bd31780ba None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.277031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eccc9f-5cdb-42ea-a802-de1bd31780ba None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.278588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 97/211] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:13.284157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.284490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.284733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.287248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eccc9f-5cdb-42ea-a802-de1bd31780ba None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:13.306276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:13.308511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:13.384312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 115/212] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:13.407689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:13.412158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:13.419982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:13.427339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eccc9f-5cdb-42ea-a802-de1bd31780ba None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:13.465763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:13.466888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.467319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.467742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.469157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:13.470758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:13.472594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 98/213] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:13.481322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:13.580218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f88c8d-9cda-4cd1-8951-4f2e8518f0af None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:13.638804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67927e1-28de-478f-81d2-973c45b53dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:13.639618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67927e1-28de-478f-81d2-973c45b53dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.639902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67927e1-28de-478f-81d2-973c45b53dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.640236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67927e1-28de-478f-81d2-973c45b53dd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.652949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67927e1-28de-478f-81d2-973c45b53dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f67927e1-28de-478f-81d2-973c45b53dd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:13.661238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:13.662118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:13.662902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 116/214] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:13.668794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20343795-887a-4b2d-8127-7d1ddac3242d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.668794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20343795-887a-4b2d-8127-7d1ddac3242d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.668794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20343795-887a-4b2d-8127-7d1ddac3242d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.669056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caaa73-76d9-4d99-89a8-bc0f9314d4ec None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:13.669144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 117/215] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:13.670399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20343795-887a-4b2d-8127-7d1ddac3242d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:13.706150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e99cc-fc8e-466d-bf6f-5e0fcc78071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:13.706831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e99cc-fc8e-466d-bf6f-5e0fcc78071e None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.707112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e99cc-fc8e-466d-bf6f-5e0fcc78071e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.707393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e99cc-fc8e-466d-bf6f-5e0fcc78071e None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.708132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e99cc-fc8e-466d-bf6f-5e0fcc78071e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:13.709586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e99cc-fc8e-466d-bf6f-5e0fcc78071e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:13.732524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 99/216] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:13 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:13.741343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:13.746288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.746573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.746851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.768998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:13.779076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:13.780593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:13.817923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20343795-887a-4b2d-8127-7d1ddac3242d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:13.869677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 100/217] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:13.875559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:13.876294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:13.876577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:13.876854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:13.877702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:13.878015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:13.879435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:13.891032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 118/218] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:13.893664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:13.901332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.042004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc95ef2-7a12-4039-a41c-09e5de366e72 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.045220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325ca9-f0ca-4db7-a617-5dc697ac942d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.079575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:14.092673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.092904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.093575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.093861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.094138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.094738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.095291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.095755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] PATH_INFO: `/v3/users/611080f3f11f4b70ad1779008080e6f8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.097387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:14.110353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:14.116255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] RBAC: Authorizing `identity:get_user(user_id=611080f3f11f4b70ad1779008080e6f8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.117969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:14.119145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:14.120214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:14.121106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 101/219] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:14.123475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 119/220] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:13 2022] GET /identity/v3/users/611080f3f11f4b70ad1779008080e6f8 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:14.126636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.128092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.249615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5c929-daa5-44be-8ac3-ee0461f82593 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.250511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4363d6-d9be-4fad-a5a3-1ccd372ea60d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.287278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.287961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.288343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.288618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.289725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:14.289725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.291360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.291813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.292498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.292643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.293259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.302036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 102/221] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:14.308083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.354059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:14.370021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:14.377878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.379585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.418977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15faedb-d5d0-489b-9cc9-f12f0589f8f1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.456191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.457128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.457414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.457690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] PATH_INFO: `/v3/users/f748783700994248bec69f5d9e1515ed` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.459092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] RBAC: Authorizing `identity:get_user(user_id=f748783700994248bec69f5d9e1515ed)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.460672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.462878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 103/222] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/users/f748783700994248bec69f5d9e1515ed => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:14.468760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.569073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 120/223] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:14.576296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.605697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edb2e-d815-423c-99c7-f5ebbc819668 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.644191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:14.659521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f2b8c-a523-4b8c-a923-20dd0477f8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.660451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f2b8c-a523-4b8c-a923-20dd0477f8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.660738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f2b8c-a523-4b8c-a923-20dd0477f8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.661015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f2b8c-a523-4b8c-a923-20dd0477f8a2 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.662239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f2b8c-a523-4b8c-a923-20dd0477f8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.663701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f2b8c-a523-4b8c-a923-20dd0477f8a2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.665810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 104/224] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:14.667767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10d3b-e356-41d2-9e21-868a9222ecab None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.668078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10d3b-e356-41d2-9e21-868a9222ecab None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.668380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10d3b-e356-41d2-9e21-868a9222ecab None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.669806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 105/225] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:14.674903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b10d3b-e356-41d2-9e21-868a9222ecab None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.680188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:14.709136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d823b1f-64d5-4c39-a948-3009c2dda7fa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.763070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708581a-8ccf-42d0-bc82-537c94976f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.763972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708581a-8ccf-42d0-bc82-537c94976f0b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.764301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708581a-8ccf-42d0-bc82-537c94976f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.764581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708581a-8ccf-42d0-bc82-537c94976f0b None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.766649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708581a-8ccf-42d0-bc82-537c94976f0b None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.768848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708581a-8ccf-42d0-bc82-537c94976f0b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.771860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 121/226] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:14.774215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.774516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.774798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.799406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:14.801616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:14.804923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b10d3b-e356-41d2-9e21-868a9222ecab None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:14.846918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:14.847959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:14.848376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:14.848755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5/users/f748783700994248bec69f5d9e1515ed/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:14.851646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] RBAC: Authorizing `identity:create_grant(project_id=8428242979624dc789b4621eb65f27a5, user_id=f748783700994248bec69f5d9e1515ed, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:14.853413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:14.871805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 106/227] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:14 2022] PUT /identity/v3/projects/8428242979624dc789b4621eb65f27a5/users/f748783700994248bec69f5d9e1515ed/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:14.872874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.907030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 122/228] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:14.941301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:14.972278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:15.028937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1cb03-5caf-4145-846e-702f7960e304 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:15.072078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:15.073502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:15.073924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:15.074223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:15.075111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=611080f3f11f4b70ad1779008080e6f8) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:15.075437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:15.077071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:15.083873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 107/229] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/role_assignments?user.id=611080f3f11f4b70ad1779008080e6f8&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:15.155497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:15.179274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c18e6a-5635-4cf2-bc2e-4c45ee2d3a29 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:15.225349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:15.226471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:15.226757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:15.227036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:15.242123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:15.250938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:15.251911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:15.252762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 123/230] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:15.261180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:15.408760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8872f-f5f1-4f35-a2ba-9e131f5a7ed0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:15.435832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:15.459724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:15.460453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:15.460736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:15.461011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:15.461767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:15.462074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:15.464081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:15.473281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 108/231] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:15.479877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:15.623501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16908f62-700a-44f2-9eb9-50d8fb78c612 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:15.668309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:15.669124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:15.669410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:15.669686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] PATH_INFO: `/v3/users/f748783700994248bec69f5d9e1515ed` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:15.671158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] RBAC: Authorizing `identity:get_user(user_id=f748783700994248bec69f5d9e1515ed)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:15.672679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:15.675354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 124/232] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:15 2022] GET /identity/v3/users/f748783700994248bec69f5d9e1515ed => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:15.680177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:15.852144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf364d6-f39f-4100-888f-d31321a9b9e9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:15.899488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5c2ff-2a1f-4b17-a77b-94aaa431cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:15.901117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5c2ff-2a1f-4b17-a77b-94aaa431cda0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:15.901401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5c2ff-2a1f-4b17-a77b-94aaa431cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:15.901762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5c2ff-2a1f-4b17-a77b-94aaa431cda0 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:15.908882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5c2ff-2a1f-4b17-a77b-94aaa431cda0 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:15.910386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5c2ff-2a1f-4b17-a77b-94aaa431cda0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:15.912191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 109/233] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:15 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:15.918930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:15.931025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:16.046373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3d845-092f-4b7b-be33-817b30dbb1b7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:16.087543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a28d10-5721-4fc8-8689-c0d16d7a7299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:16.089054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a28d10-5721-4fc8-8689-c0d16d7a7299 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:16.089341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a28d10-5721-4fc8-8689-c0d16d7a7299 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:16.089618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a28d10-5721-4fc8-8689-c0d16d7a7299 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5/users/f748783700994248bec69f5d9e1515ed/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:16.097373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a28d10-5721-4fc8-8689-c0d16d7a7299 None admin] RBAC: Authorizing `identity:create_grant(project_id=8428242979624dc789b4621eb65f27a5, user_id=f748783700994248bec69f5d9e1515ed, role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:16.098843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a28d10-5721-4fc8-8689-c0d16d7a7299 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:16.126124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 125/234] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:15 2022] PUT /identity/v3/projects/8428242979624dc789b4621eb65f27a5/users/f748783700994248bec69f5d9e1515ed/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:16.565108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546b0c6-63a7-4475-89f4-05ecf2ee8d6a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:16.565618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546b0c6-63a7-4475-89f4-05ecf2ee8d6a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:16.565692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546b0c6-63a7-4475-89f4-05ecf2ee8d6a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:16.567072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 110/235] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:16.584899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172e7e4-8916-40bb-a5af-44f5443dc084 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:16.585407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172e7e4-8916-40bb-a5af-44f5443dc084 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:16.585627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172e7e4-8916-40bb-a5af-44f5443dc084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:16.613373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b172e7e4-8916-40bb-a5af-44f5443dc084 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:16.614639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b172e7e4-8916-40bb-a5af-44f5443dc084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:16.707230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:16.718819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 126/236] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:16.731926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bc98d-f839-4112-9f93-2cf2931957d3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:16.732461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bc98d-f839-4112-9f93-2cf2931957d3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:16.732881 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bc98d-f839-4112-9f93-2cf2931957d3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:16.734648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 111/237] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:16.740319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:16.740614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:16.740894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:16.749191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bc98d-f839-4112-9f93-2cf2931957d3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:16.769199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:16.770905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:16.848985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:16.884213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 127/238] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:16.925509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:16.940496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bc98d-f839-4112-9f93-2cf2931957d3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.012726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.013540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.013822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.014096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.035545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:17.044977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:17.045913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:17.046715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:17.049130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 112/239] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:17.054718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:17.135732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb54aa-8b4a-425c-876e-46533b8cab71 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.198400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:17.219030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.220343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.220867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.221340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.222770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:17.224490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:17.229496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 128/240] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:17.235210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:17.276767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52833416-82e3-4c1f-b947-6a50fed45add None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.297356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:17.355558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.356293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.356579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.356857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.357622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:17.357935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:17.359341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:17.376410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 113/241] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:17.381536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:17.455421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f58c3-96a3-4e58-ba76-dd03edeaeda1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.521081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae101d-ffbf-490c-9563-7592f3c1776a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.540034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7c24-e555-45ca-b5a8-79d626ea103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.540844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7c24-e555-45ca-b5a8-79d626ea103f None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.541107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7c24-e555-45ca-b5a8-79d626ea103f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.541386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7c24-e555-45ca-b5a8-79d626ea103f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.542135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae7c24-e555-45ca-b5a8-79d626ea103f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:17.543729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae7c24-e555-45ca-b5a8-79d626ea103f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:17.579540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.580387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.580675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.580953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] PATH_INFO: `/v3/users/603755da45d645f7920ea73a3b16f4d3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.590679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 129/242] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:17 2022] POST /identity/v3/users => generated 293 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:17.594731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc5086-0c09-4e4f-9b68-9ef1111bfdad None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.594731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc5086-0c09-4e4f-9b68-9ef1111bfdad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.594731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc5086-0c09-4e4f-9b68-9ef1111bfdad None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.594731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 130/243] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:17.603745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] RBAC: Authorizing `identity:get_user(user_id=603755da45d645f7920ea73a3b16f4d3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:17.605303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:17.605576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.605873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.606157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.607115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 114/244] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:17 2022] GET /identity/v3/users/603755da45d645f7920ea73a3b16f4d3 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:17.612984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:17.644955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:17.646217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:17.682873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:17.747369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:17.764785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 131/245] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:17.787478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:17.861775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302c37-fdcf-4961-b500-cdabbfe1256b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.912687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.914170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.915199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.916615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.918656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20d773-3638-4d82-8918-c50c4efe4596 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:17.965361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c934e8-4364-469f-abe5-f29be6f25271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:17.966946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c934e8-4364-469f-abe5-f29be6f25271 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.966946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c934e8-4364-469f-abe5-f29be6f25271 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.966946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c934e8-4364-469f-abe5-f29be6f25271 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.980763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c934e8-4364-469f-abe5-f29be6f25271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-75c934e8-4364-469f-abe5-f29be6f25271 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:17.989679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:17.990613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:17.991436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 132/246] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:17.992128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d285-63d6-4dcd-92f3-605849064a0b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:17.992702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d285-63d6-4dcd-92f3-605849064a0b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:17.992702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d285-63d6-4dcd-92f3-605849064a0b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:17.994011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 133/247] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:18.000521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.000973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.001406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.020402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:18.021561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:18.021693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:18.023135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:18.060879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:18.068850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:18.115172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 134/248] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:18.116383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.152926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 115/249] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:18.153796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.258914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0553d-b2bb-494b-af04-37b0e82c1e94 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:18.298325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2c4ef-7a24-4b6e-998c-3ca62dea2a0e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:18.308225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c9108-78b1-4980-8699-2868c444accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:18.308937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9108-78b1-4980-8699-2868c444accd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.309216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9108-78b1-4980-8699-2868c444accd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.309487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9108-78b1-4980-8699-2868c444accd None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.310275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c9108-78b1-4980-8699-2868c444accd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:18.310582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c9108-78b1-4980-8699-2868c444accd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:18.312012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c9108-78b1-4980-8699-2868c444accd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:18.321633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 135/250] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:18.323172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c9108-78b1-4980-8699-2868c444accd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.349663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:18.364164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:18.368372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:18.369485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.369917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.370902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.394294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:18.421225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:18.422187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:18.422977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 116/251] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:18.423617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.465866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c9108-78b1-4980-8699-2868c444accd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:18.514322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:18.514322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.514322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.514322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.517720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:18.519371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:18.521324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 136/252] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:18.522595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.683106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58bd60-146a-488c-9bc9-8fd7d8d63a72 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:18.686696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:18.705253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ab016-770b-4953-b57f-364d7b30f8ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:18.761208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:18.761955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.762236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.762511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.763335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:18.763639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:18.765085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:18.777127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:18.778018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.778298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.778574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] PATH_INFO: `/v3/users/f748783700994248bec69f5d9e1515ed` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.780088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] RBAC: Authorizing `identity:get_user(user_id=f748783700994248bec69f5d9e1515ed)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:18.781558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:18.783194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 117/253] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/users/f748783700994248bec69f5d9e1515ed => generated 283 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:18.787850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 137/254] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:18.788385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.790333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:18.880495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:18.919979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519739a-be61-4da7-9ad5-b6b98e978c77 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:18.960284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:18.960874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:18.961220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:18.961499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:18.962410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] RBAC: Adding query filter params (role.id=a1a7ce3d503044739a4c81a7fba52623, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=603755da45d645f7920ea73a3b16f4d3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:18.962758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:18.964527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:18.971042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 118/255] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/role_assignments?user.id=603755da45d645f7920ea73a3b16f4d3&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=a1a7ce3d503044739a4c81a7fba52623 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:18.972219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.075228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1af23e-5661-4706-a8c7-586b853314ba None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:19.104200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77f104-aa7c-4dfd-8920-f229ed54798a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:19.121611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:19.139895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd76d-8a35-43ce-add7-3220e4c5a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:19.140868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd76d-8a35-43ce-add7-3220e4c5a697 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.141189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd76d-8a35-43ce-add7-3220e4c5a697 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.141382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd76d-8a35-43ce-add7-3220e4c5a697 None admin] PATH_INFO: `/v3/users/611080f3f11f4b70ad1779008080e6f8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.142665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdd76d-8a35-43ce-add7-3220e4c5a697 None admin] RBAC: Authorizing `identity:get_user(user_id=611080f3f11f4b70ad1779008080e6f8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:19.144182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdd76d-8a35-43ce-add7-3220e4c5a697 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:19.145875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 119/256] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/users/611080f3f11f4b70ad1779008080e6f8 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:19.151296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5974f1b-761f-4c9e-ac84-0f81fb9f9df5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.151296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5974f1b-761f-4c9e-ac84-0f81fb9f9df5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.151296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5974f1b-761f-4c9e-ac84-0f81fb9f9df5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.151296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 120/257] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:19.155152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:19.156181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.156383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5974f1b-761f-4c9e-ac84-0f81fb9f9df5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.157003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.157492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.201369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:19.263271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:19.265280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:19.271340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5974f1b-761f-4c9e-ac84-0f81fb9f9df5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:19.311251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc570-5748-420f-9246-8af38f6f78df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:19.312155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc570-5748-420f-9246-8af38f6f78df None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.312439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc570-5748-420f-9246-8af38f6f78df None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.312712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc570-5748-420f-9246-8af38f6f78df None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.314023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc570-5748-420f-9246-8af38f6f78df None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:19.315377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cc570-5748-420f-9246-8af38f6f78df None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:19.322013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 121/258] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:19 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:19.322013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.322013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.322013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.339610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:19.341048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:19.393948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 138/259] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:19.395506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.398575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:19.418330 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 122/260] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:19.419275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.480096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:19.559270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b57a3-941b-437b-b0bd-c50238f06778 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:19.564171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3575b-9fd7-416f-880a-8d59a66e5c4e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:19.613995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:19.614858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.615137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.615408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.616757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:19.618131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:19.624117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 123/261] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:19 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:19.624117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.652819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:19.653723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.654009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.654550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5/users/611080f3f11f4b70ad1779008080e6f8/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.664933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8428242979624dc789b4621eb65f27a5, user_id=611080f3f11f4b70ad1779008080e6f8, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:19.667044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:19.704561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:19.715100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 139/262] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:19 2022] PUT /identity/v3/projects/8428242979624dc789b4621eb65f27a5/users/611080f3f11f4b70ad1779008080e6f8/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:19.715966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.745366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa325b-9958-4f03-b2e6-47f46c4c51ef None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:19.839971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:19.841556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:19.841842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:19.842113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:19.871115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:19.880876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:19.882366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:19.883358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 124/263] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:19.890957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:19.998040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f5aa-9151-4a77-bda5-85283c4b59f6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:20.029148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:20.049697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:20.050611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:20.051034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:20.051449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:20.052524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=f748783700994248bec69f5d9e1515ed) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:20.053010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:20.054618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:20.068533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 140/264] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:19 2022] GET /identity/v3/role_assignments?user.id=f748783700994248bec69f5d9e1515ed&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:20.072973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a95577-fb7c-4808-ab52-e4d7ef854580 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:20.113028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:20.113721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:20.114006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:20.114284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:20.115124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:20.115444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:20.115941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:20.121325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:20.131709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 125/265] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:20.141253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:20.284567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83a2ed-5f7d-4ebf-b390-abd5f3706752 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:20.331695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:20.332977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:20.333460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:20.333950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] PATH_INFO: `/v3/users/f748783700994248bec69f5d9e1515ed` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:20.335426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] RBAC: Authorizing `identity:get_user(user_id=f748783700994248bec69f5d9e1515ed)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:20.337105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:20.340036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 141/266] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:20 2022] GET /identity/v3/users/f748783700994248bec69f5d9e1515ed => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:20.345836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:20.547379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d793-5c97-4a4d-af0d-b4f67d87e32b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:20.599435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ccba5-740d-480c-bf0a-cd0a9fe0f659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:20.600146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ccba5-740d-480c-bf0a-cd0a9fe0f659 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:20.600435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ccba5-740d-480c-bf0a-cd0a9fe0f659 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:20.600715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ccba5-740d-480c-bf0a-cd0a9fe0f659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:20.668090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:20.702452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ccba5-740d-480c-bf0a-cd0a9fe0f659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:20.704574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ccba5-740d-480c-bf0a-cd0a9fe0f659 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:20.850154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 126/267] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:20.857137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:20.899874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1672af-f64d-4b00-9f02-896a084999dd None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:20.900683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1672af-f64d-4b00-9f02-896a084999dd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:20.901158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1672af-f64d-4b00-9f02-896a084999dd None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:20.902767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 127/268] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:20.910921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:20.911547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:20.912149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:20.940250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:20.943254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:20.969874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:21.025087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce3139-2bf8-4762-b422-1f3fd84b4b5f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.067430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7104335-2699-4cb8-abb3-daea1929977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.068345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7104335-2699-4cb8-abb3-daea1929977f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.068633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7104335-2699-4cb8-abb3-daea1929977f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.068911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7104335-2699-4cb8-abb3-daea1929977f None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.070157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7104335-2699-4cb8-abb3-daea1929977f None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:21.071619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7104335-2699-4cb8-abb3-daea1929977f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:21.078151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 142/269] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:20 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:21.079824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992565d2-218c-43f7-9d38-73202595aa13 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.080316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992565d2-218c-43f7-9d38-73202595aa13 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.080719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992565d2-218c-43f7-9d38-73202595aa13 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.082638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 143/270] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:21.089517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.090145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.090617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.110245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 128/271] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:21.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:21.115838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:21.117503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:21.121913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:21.214758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 144/272] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:21.215953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:21.268746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e0610-ba75-41bb-94ce-8ef5d6f9a580 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.292874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:21.311768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.312825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.313411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.313879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.314943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] RBAC: Adding query filter params (role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=f748783700994248bec69f5d9e1515ed) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:21.315462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:21.317127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:21.324772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 129/273] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:21 2022] GET /identity/v3/role_assignments?user.id=f748783700994248bec69f5d9e1515ed&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:21.325883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:21.388353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65548f1b-5cd3-456b-83d3-eac892de1dcf None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.421264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:21.453549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.454440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.454881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.455077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.469005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:21.469785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891915c-866f-44a7-b576-3792dcf6784d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:21.480086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:21.481066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:21.481854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 145/274] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:21.486774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:21.542790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.543626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.543909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.544214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.546564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:21.550655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:21.553504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 130/275] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:21.562148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:21.675567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71e2be-c799-43dd-9782-88b86489ff22 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.697339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:21.725121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cbda5-a839-4cc5-bfb6-fed9f5c1413e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.727626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.729115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.729638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.730085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.732765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:21.733275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:21.734954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:21.747342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 146/276] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:21.754242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:21.782595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcf680-7007-4f7e-a41a-d3bceab8ae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.783662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcf680-7007-4f7e-a41a-d3bceab8ae27 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.784255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcf680-7007-4f7e-a41a-d3bceab8ae27 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.784766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcf680-7007-4f7e-a41a-d3bceab8ae27 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.785730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcf680-7007-4f7e-a41a-d3bceab8ae27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:21.787718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcf680-7007-4f7e-a41a-d3bceab8ae27 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:21.800694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:21.823349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 131/277] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:21 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:21.887476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dce39-d19c-477b-b0f2-582e1c4e35b3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.888625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dce39-d19c-477b-b0f2-582e1c4e35b3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.889215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dce39-d19c-477b-b0f2-582e1c4e35b3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.891265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 132/278] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:21.891984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aaa8c-483c-4cca-9be9-16f60fabdc96 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:21.898127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.898762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.899276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.925039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:21.926726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:21.933945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:21.934867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:21.935148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:21.935420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] PATH_INFO: `/v3/users/59c860a9ae8a4d849eac614112fdf357` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:21.952150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] RBAC: Authorizing `identity:get_user(user_id=59c860a9ae8a4d849eac614112fdf357)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:21.953651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:21.956083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 147/279] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:21 2022] GET /identity/v3/users/59c860a9ae8a4d849eac614112fdf357 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:21.962182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.002238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:22.018320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 133/280] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:22.042314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.092577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb492f-f3bd-49a2-8b1d-9a4d75a38306 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:22.146219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:22.146936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.147218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.147493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.148509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, user.id=59c860a9ae8a4d849eac614112fdf357, scope.system=all) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:22.148827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:22.150958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:22.157916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 148/281] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Jun 16 18:14:21 2022] GET /identity/v3/role_assignments?user.id=59c860a9ae8a4d849eac614112fdf357&scope.system=all&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:22.192599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9b1ab-cf76-4dce-a27c-bd8b3ce5fd6a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:22.247496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:22.272401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8b310-d390-426d-8269-318ffc1bbdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:22.273769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b310-d390-426d-8269-318ffc1bbdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.274736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b310-d390-426d-8269-318ffc1bbdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.276302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b310-d390-426d-8269-318ffc1bbdb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.298644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8b310-d390-426d-8269-318ffc1bbdb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-55e8b310-d390-426d-8269-318ffc1bbdb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:22.314498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:22.315468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:22.316276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:22.317667 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 134/282] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:22.324807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.448922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:22.453578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a0544-9bf8-4947-b7b4-4940f63bb626 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:22.501435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:22.502143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.502428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.502708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.503570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:22.505275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19132e7-6318-46db-92ea-6163d60734dc None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.505560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19132e7-6318-46db-92ea-6163d60734dc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.505835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19132e7-6318-46db-92ea-6163d60734dc None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.507362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 135/283] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:22.507524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:22.509134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:22.513731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.514025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.514297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.537980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 149/284] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:22.539024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:22.540266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:22.543160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.691212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 136/285] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:22.699912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd29519-44ea-4f61-95af-2de5c976b187 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:22.726738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.742129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:22.743040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.743315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.743588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] PATH_INFO: `/v3/users/603755da45d645f7920ea73a3b16f4d3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.748291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] RBAC: Authorizing `identity:get_user(user_id=603755da45d645f7920ea73a3b16f4d3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:22.749770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:22.751680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 150/286] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:22 2022] GET /identity/v3/users/603755da45d645f7920ea73a3b16f4d3 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:22.757207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.828453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:22.865082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945278-219e-4983-bf4a-909e661fa914 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:22.882656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:22.907455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:22.908290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:22.908494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:22.908760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:22.921763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:22.935365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:22.936353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:22.937154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 137/287] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:22.943326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:22.996349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7638c-290d-440b-9fa2-d44a6c6201b6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:23.049291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:23.052069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:23.052069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:23.052069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:23.054442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:23.056074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:23.060666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 151/288] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:22 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:23.067566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:23.192304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918ee47-0f9c-47a0-8860-9c797532b741 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:23.282191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595751d-d823-4304-a788-47e169f28519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:23.282806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595751d-d823-4304-a788-47e169f28519 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:23.283088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595751d-d823-4304-a788-47e169f28519 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:23.283364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595751d-d823-4304-a788-47e169f28519 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:23.284151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595751d-d823-4304-a788-47e169f28519 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:23.284612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595751d-d823-4304-a788-47e169f28519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:23.286424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595751d-d823-4304-a788-47e169f28519 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:23.296918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:23.299160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 138/289] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:23.304068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70c9f-74b1-4dcc-a113-08321f7d431c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:23.305247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595751d-d823-4304-a788-47e169f28519 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:23.341934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:23.342846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:23.343133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:23.343413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5/users/603755da45d645f7920ea73a3b16f4d3/roles/a1a7ce3d503044739a4c81a7fba52623` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:23.345521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] RBAC: Authorizing `identity:create_grant(project_id=8428242979624dc789b4621eb65f27a5, user_id=603755da45d645f7920ea73a3b16f4d3, role_id=a1a7ce3d503044739a4c81a7fba52623)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:23.347009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:23.370561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 152/290] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:14:23 2022] PUT /identity/v3/projects/8428242979624dc789b4621eb65f27a5/users/603755da45d645f7920ea73a3b16f4d3/roles/a1a7ce3d503044739a4c81a7fba52623 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:23.396316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:23.467378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595751d-d823-4304-a788-47e169f28519 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:23.522804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:23.523836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:23.524152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:23.524439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] PATH_INFO: `/v3/users/611080f3f11f4b70ad1779008080e6f8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:23.528908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] RBAC: Authorizing `identity:get_user(user_id=611080f3f11f4b70ad1779008080e6f8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:23.530657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:23.532706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 139/291] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:23 2022] GET /identity/v3/users/611080f3f11f4b70ad1779008080e6f8 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:23.538206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:23.677187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f0f32-8af0-4eec-9422-77bedfcc2558 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:23.715053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:23.715717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:23.715997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:23.716303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:23.747530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:23.802380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:23.803780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:23.943626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 153/292] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:23.949791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:24.101004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802300bb-84c8-479d-ae65-37b265c31695 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:24.105117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:24.147220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209cf88-29bc-484c-8642-7c484a65710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:24.148484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209cf88-29bc-484c-8642-7c484a65710e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:24.148988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209cf88-29bc-484c-8642-7c484a65710e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:24.149457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209cf88-29bc-484c-8642-7c484a65710e None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:24.156941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209cf88-29bc-484c-8642-7c484a65710e None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:24.158724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209cf88-29bc-484c-8642-7c484a65710e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:24.161068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 140/293] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:23 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:24.167599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:24.253233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:24.302436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72886e6a-6633-43d7-9ac4-536dbfa7b402 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:24.344348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:24.345025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:24.345307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:24.345586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:24.346443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=611080f3f11f4b70ad1779008080e6f8) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:24.346755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:24.348232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098fed-19b3-4f4f-bceb-2d836e329271 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:24.359434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 154/294] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:24 2022] GET /identity/v3/role_assignments?user.id=611080f3f11f4b70ad1779008080e6f8&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:24.665740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce4e5-791d-424f-a388-756e7cf6f9da None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:24.667243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce4e5-791d-424f-a388-756e7cf6f9da None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:24.667905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce4e5-791d-424f-a388-756e7cf6f9da None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:24.669916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 141/295] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:24.675240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:24.675861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:24.676360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:24.700679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:24.702414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:24.773785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a537e-a5a6-4916-a863-39633d18183f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:24.774111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a537e-a5a6-4916-a863-39633d18183f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:24.774344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a537e-a5a6-4916-a863-39633d18183f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:24.776888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 142/296] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:24.782293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:24.782706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:24.782883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:24.803928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:24.804556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 155/297] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:24.805157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:24.830636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:24.905455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:24.906060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 143/298] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:24.934091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:24.961058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75b3ba-2fb5-4929-84eb-08b0ac7c55d3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:24.999961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.000810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.001467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.001811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.014271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:25.022551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:25.023526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:25.024899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 156/299] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:25.032717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.062314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929cbe9-7a41-4981-82bf-b196f560c2d9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.064794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:25.110366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608598-99bc-4903-b8de-a311edddfab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.111265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608598-99bc-4903-b8de-a311edddfab5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.111823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608598-99bc-4903-b8de-a311edddfab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.111946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608598-99bc-4903-b8de-a311edddfab5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.124395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad608598-99bc-4903-b8de-a311edddfab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ad608598-99bc-4903-b8de-a311edddfab5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:25.132690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:25.133604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:25.134396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 144/300] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:25.136577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a4b0c-d307-4ef9-b32a-3e51f08544c0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.136577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a4b0c-d307-4ef9-b32a-3e51f08544c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.136715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a4b0c-d307-4ef9-b32a-3e51f08544c0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.138040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 145/301] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:25.141608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a4b0c-d307-4ef9-b32a-3e51f08544c0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.177994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4287d-7caa-4265-a7a0-9f08ea9bada8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.190940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:25.218899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.219578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.219861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.220167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.220934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:25.221248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.222625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9828509-e1fd-42f9-a6cb-2289f8e699c3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.232411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 157/302] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:25.233599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.233892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.234170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.252381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:25.253580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:25.265372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a4b0c-d307-4ef9-b32a-3e51f08544c0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.305635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.306311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.307126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.307520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.308303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:25.308613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.309991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.325129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 146/303] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:25.325129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.343977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 158/304] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:25.352321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:25.353094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.438347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170d35a-1f18-4a07-b42a-f4bd337f130a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.472920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.473778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.474081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.474355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] PATH_INFO: `/v3/users/ebd9c03aaac44fea8a4f1bcef637b760` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.475911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f702e1-36da-423a-bdf5-0db271fd86b2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.478781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:25.490949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd9c03aaac44fea8a4f1bcef637b760)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.492838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.499458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 147/305] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:25 2022] GET /identity/v3/users/ebd9c03aaac44fea8a4f1bcef637b760 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:25.499458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.528195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.529079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.529875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.529875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] PATH_INFO: `/v3/users/59c860a9ae8a4d849eac614112fdf357` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.531081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] RBAC: Authorizing `identity:get_user(user_id=59c860a9ae8a4d849eac614112fdf357)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.532550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.538699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 159/306] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:25 2022] GET /identity/v3/users/59c860a9ae8a4d849eac614112fdf357 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:25.538699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.601169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:25.626445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4787-a0e8-4ff5-a498-36ac2493ea00 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.650781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba9d85-6dd0-49b0-b301-d291d9ffdd5c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.664661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.665477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.665785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.666596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.668514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.670074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.671769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 148/307] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:25.672542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.697086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.698005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.698906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.699255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.700251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] RBAC: Adding query filter params (role.id=a1a7ce3d503044739a4c81a7fba52623, user.id=ebd9c03aaac44fea8a4f1bcef637b760, scope.system=all) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:25.700640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.702196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.708153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 160/308] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Jun 16 18:14:25 2022] GET /identity/v3/role_assignments?user.id=ebd9c03aaac44fea8a4f1bcef637b760&scope.system=all&role.id=a1a7ce3d503044739a4c81a7fba52623 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:25.709089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:25.764850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:25.806148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dcb82-c6a4-4f0f-9a53-78c3dea5911e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.862794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1f73d-4cb2-434a-a76d-8bd02a1326e5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:25.863066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67dfd5-8985-42c3-b13a-71d0c1dc930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.864196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67dfd5-8985-42c3-b13a-71d0c1dc930b None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.864560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67dfd5-8985-42c3-b13a-71d0c1dc930b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.864915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67dfd5-8985-42c3-b13a-71d0c1dc930b None admin] PATH_INFO: `/v3/system/users/59c860a9ae8a4d849eac614112fdf357/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.868351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67dfd5-8985-42c3-b13a-71d0c1dc930b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=59c860a9ae8a4d849eac614112fdf357, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.869914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67dfd5-8985-42c3-b13a-71d0c1dc930b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.892204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 149/309] 10.222.0.30 () {58 vars in 1334 bytes} [Thu Jun 16 18:14:25 2022] PUT /identity/v3/system/users/59c860a9ae8a4d849eac614112fdf357/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:25.920584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186caf7a-6743-40f6-ad39-ad14b09cd066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:25.921684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186caf7a-6743-40f6-ad39-ad14b09cd066 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.922119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186caf7a-6743-40f6-ad39-ad14b09cd066 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.922605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186caf7a-6743-40f6-ad39-ad14b09cd066 None admin] PATH_INFO: `/v3/groups` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.923519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186caf7a-6743-40f6-ad39-ad14b09cd066 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:25.925394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186caf7a-6743-40f6-ad39-ad14b09cd066 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:25.947641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 161/310] 10.222.0.30 () {60 vars in 1134 bytes} [Thu Jun 16 18:14:25 2022] POST /identity/v3/groups => generated 229 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:25.990837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be14b-0abe-4d71-891b-36eef75352eb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:25.991400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be14b-0abe-4d71-891b-36eef75352eb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:25.991838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be14b-0abe-4d71-891b-36eef75352eb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:25.993923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 150/311] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:25.999792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:26.000351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:26.000791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:26.020033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:26.021611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:26.106510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 162/312] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:26.130949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6be14b-0abe-4d71-891b-36eef75352eb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:26.150223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:26.249424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6be14b-0abe-4d71-891b-36eef75352eb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:26.287261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:26.288388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:26.288875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:26.289331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:26.302214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:26.310590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:26.311527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:26.312356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:26.318424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 151/313] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:26.321376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:26.348631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:26.435179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ddae3-8e64-494f-b0da-9a728d210f30 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:26.481402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:26.482481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:26.482969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:26.483431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:26.484453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:26.484974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:26.486635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:26.498222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 163/314] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:26.504092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:26.600528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:26.630761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5968f-5d6c-4373-a732-da480878e589 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:26.673251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:26.674544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:26.675038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:26.675527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] PATH_INFO: `/v3/users/603755da45d645f7920ea73a3b16f4d3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:26.677119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] RBAC: Authorizing `identity:get_user(user_id=603755da45d645f7920ea73a3b16f4d3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:26.678822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:26.686059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 152/315] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:26 2022] GET /identity/v3/users/603755da45d645f7920ea73a3b16f4d3 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:26.689181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:26.793934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:26.816040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea764aa-3c57-4c41-a93c-6ee4ba06d7c2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:26.853326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:26.854376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:26.854834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:26.855124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:26.934310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:26.935798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:27.028599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:27.043315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 164/316] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:27.049387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:27.156709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d82b7-647f-4a8d-a995-8d58f802c129 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:27.195828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a85ff-0bc3-431f-a841-ece3d0e6a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:27.198030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a85ff-0bc3-431f-a841-ece3d0e6a713 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:27.198547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a85ff-0bc3-431f-a841-ece3d0e6a713 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:27.199053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a85ff-0bc3-431f-a841-ece3d0e6a713 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:27.200716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a85ff-0bc3-431f-a841-ece3d0e6a713 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:27.202490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a85ff-0bc3-431f-a841-ece3d0e6a713 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:27.210476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 153/317] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:27 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:27.212150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:27.321076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59afea9-19dd-443d-9546-2cd347a7b7a9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:27.357488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:27.358301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:27.358490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:27.358772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:27.359667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] RBAC: Adding query filter params (role.id=a1a7ce3d503044739a4c81a7fba52623, scope.project.id=8428242979624dc789b4621eb65f27a5, user.id=603755da45d645f7920ea73a3b16f4d3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:27.359983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:27.361454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2fe3f-74a0-42d0-9066-ceb5273c2ba6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:27.368190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 165/318] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:14:27 2022] GET /identity/v3/role_assignments?user.id=603755da45d645f7920ea73a3b16f4d3&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=a1a7ce3d503044739a4c81a7fba52623 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:27.959403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5174-9f9d-46cc-bf90-65f5ffbeceff None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:27.959800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5174-9f9d-46cc-bf90-65f5ffbeceff None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:27.960009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f5174-9f9d-46cc-bf90-65f5ffbeceff None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:27.961543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 154/319] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:27.966832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:27.967127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:27.967405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:27.984538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:27.985726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:28.035739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.059666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 166/320] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:28.081629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f5174-9f9d-46cc-bf90-65f5ffbeceff None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.095576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.196564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f5174-9f9d-46cc-bf90-65f5ffbeceff None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.239066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e4ae3-1acf-4e37-aa7f-20574724b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.239812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e4ae3-1acf-4e37-aa7f-20574724b599 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.240119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e4ae3-1acf-4e37-aa7f-20574724b599 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.240399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e4ae3-1acf-4e37-aa7f-20574724b599 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.253592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e4ae3-1acf-4e37-aa7f-20574724b599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-461e4ae3-1acf-4e37-aa7f-20574724b599 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:28.261802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:28.262695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:28.264622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 155/321] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:28.269339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.287280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ef73a-3e94-4332-9dba-bd3263370d20 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.287575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ef73a-3e94-4332-9dba-bd3263370d20 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.287850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ef73a-3e94-4332-9dba-bd3263370d20 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.289341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 156/322] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:28.296213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ecfde-19ca-4e01-971b-dc92cc091adf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.296657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ecfde-19ca-4e01-971b-dc92cc091adf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.296837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ecfde-19ca-4e01-971b-dc92cc091adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.316776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d63ecfde-19ca-4e01-971b-dc92cc091adf None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:28.318034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d63ecfde-19ca-4e01-971b-dc92cc091adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:28.389735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.390685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd874f-ee87-47f9-92ff-f16a8cfca8b9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.396092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 157/323] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:28.396520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd1c22-0863-4f07-8cde-4984a26add76 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.396812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd1c22-0863-4f07-8cde-4984a26add76 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.397091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd1c22-0863-4f07-8cde-4984a26add76 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.398506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 158/324] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:28.404898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.405192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.405480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.423745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:28.424958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:28.429196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.429874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.430158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.430434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.431204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:28.431637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:28.433571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:28.442502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 167/325] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:28.443368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.488114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.504339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 159/326] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:28.504339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.552155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693cf47-47b0-4273-a49a-e1b61e5974ab None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.590060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.590890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.591175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.591447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.603467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:28.611335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:28.612677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 168/327] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:28.613492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.629679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138b145-00a4-4e1c-b87f-0283e3d1314d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.662220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.669804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.670717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.670996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.671269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] PATH_INFO: `/v3/users/ebd9c03aaac44fea8a4f1bcef637b760` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.672569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] RBAC: Authorizing `identity:get_user(user_id=ebd9c03aaac44fea8a4f1bcef637b760)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:28.674073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:28.676023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 160/328] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/users/ebd9c03aaac44fea8a4f1bcef637b760 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:28.676635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.732911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335bfa5-b8c3-4869-9c67-d281aba96f6c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.734624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.773262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.774010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.774296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.774576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.779069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:28.780855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:28.782546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 169/329] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:28.783301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.788707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b74ad-cb1d-4beb-a3a2-5192f28e47cd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.825242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.825925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.826210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.826483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.827238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:28.827545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:28.828951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:28.837310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 161/330] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:28.839342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.896168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea96bb-3d15-47cb-9880-f9702ae31de3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.904038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:28.931740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:28.932638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:28.932917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:28.933186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] PATH_INFO: `/v3/system/users/ebd9c03aaac44fea8a4f1bcef637b760/roles/a1a7ce3d503044739a4c81a7fba52623` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:28.934698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ebd9c03aaac44fea8a4f1bcef637b760, role_id=a1a7ce3d503044739a4c81a7fba52623)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:28.936094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:28.944519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 170/331] 10.222.0.30 () {58 vars in 1334 bytes} [Thu Jun 16 18:14:28 2022] PUT /identity/v3/system/users/ebd9c03aaac44fea8a4f1bcef637b760/roles/a1a7ce3d503044739a4c81a7fba52623 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:28.957756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:28.961443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a07bd-27a6-45d8-af7e-75eff6f9eb28 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:28.988440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:29.007231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:29.008113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:29.008398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:29.008672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] PATH_INFO: `/v3/groups` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:29.009463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:29.011522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:29.031892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 162/332] 10.222.0.30 () {60 vars in 1134 bytes} [Thu Jun 16 18:14:28 2022] POST /identity/v3/groups => generated 226 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:29.077721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa24af5-f246-4f56-a7fe-84e65cbcf0c1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:29.121044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed847661-1974-4b8e-9d3f-783f49868eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:29.122538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed847661-1974-4b8e-9d3f-783f49868eba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:29.122538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed847661-1974-4b8e-9d3f-783f49868eba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:29.122538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed847661-1974-4b8e-9d3f-783f49868eba None admin] PATH_INFO: `/v3/users/59c860a9ae8a4d849eac614112fdf357` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:29.123848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed847661-1974-4b8e-9d3f-783f49868eba None admin] RBAC: Authorizing `identity:get_user(user_id=59c860a9ae8a4d849eac614112fdf357)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:29.125383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed847661-1974-4b8e-9d3f-783f49868eba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:29.127381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 171/333] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:28 2022] GET /identity/v3/users/59c860a9ae8a4d849eac614112fdf357 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:29.133293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:29.255897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa0473-fd69-4427-8d86-a7143c109c23 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:29.280094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:29.298429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:29.299034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:29.299316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:29.299675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:29.300731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] RBAC: Adding query filter params (role.id=f21b9c85de034e5fac7761764c96d2f6, user.id=59c860a9ae8a4d849eac614112fdf357, scope.system=all) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:29.301045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:29.302545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144cd82-cebe-4f47-86ec-4eeee2e26b96 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:29.316602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 163/334] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Jun 16 18:14:29 2022] GET /identity/v3/role_assignments?user.id=59c860a9ae8a4d849eac614112fdf357&scope.system=all&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:31.390847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27d2d-ac3d-43e7-848c-6e6e95a2a84b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:31.391874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27d2d-ac3d-43e7-848c-6e6e95a2a84b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:31.392425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27d2d-ac3d-43e7-848c-6e6e95a2a84b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:31.394299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 172/335] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:31.401883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:31.402831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:31.403317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:31.426924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:31.432857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:31.522166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 164/336] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:31.562201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27d2d-ac3d-43e7-848c-6e6e95a2a84b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:31.688208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27d2d-ac3d-43e7-848c-6e6e95a2a84b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:31.689819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:31.727757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c2b5-7fe0-436c-8c64-7e068b224059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:31.729130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c2b5-7fe0-436c-8c64-7e068b224059 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:31.729416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c2b5-7fe0-436c-8c64-7e068b224059 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:31.729679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c2b5-7fe0-436c-8c64-7e068b224059 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:31.745245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c2b5-7fe0-436c-8c64-7e068b224059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1ac2c2b5-7fe0-436c-8c64-7e068b224059 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:31.753208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:31.754187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:31.754983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 173/337] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:31.761404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:31.877327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb530c-1ff5-4982-8d22-b98a3f227791 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:31.888606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:31.894028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81993-0bfc-4a22-9dc0-840719b15796 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:31.894339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81993-0bfc-4a22-9dc0-840719b15796 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:31.894577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81993-0bfc-4a22-9dc0-840719b15796 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:31.896511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 174/338] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:31.908613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:31.908971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:31.909203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:31.927926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:31.928795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:31.929075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:31.929348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:31.930132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:31.930476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:31.932947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:31.943699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 165/339] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:31.949617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:31.950144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:31.951414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:32.074247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5102c-e480-4a3a-a5a1-73c630a594a9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:32.117206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:32.118405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.118836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.119947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] PATH_INFO: `/v3/users/ebd9c03aaac44fea8a4f1bcef637b760` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.121556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd9c03aaac44fea8a4f1bcef637b760)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:32.123191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:32.125497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 166/340] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:14:31 2022] GET /identity/v3/users/ebd9c03aaac44fea8a4f1bcef637b760 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:32.132416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:32.133766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 175/341] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:32.180338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:32.184702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:32.246771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c1b8e-6ab1-4c0d-953b-75d37bf08aa6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:32.284123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:32.285068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.285492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.285913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.286897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] RBAC: Adding query filter params (role.id=a1a7ce3d503044739a4c81a7fba52623, user.id=ebd9c03aaac44fea8a4f1bcef637b760, scope.system=all) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:32.287422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:32.289091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76626436-15f9-4297-8267-55c5c9787651 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:32.303535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 167/342] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Jun 16 18:14:32 2022] GET /identity/v3/role_assignments?user.id=ebd9c03aaac44fea8a4f1bcef637b760&scope.system=all&role.id=a1a7ce3d503044739a4c81a7fba52623 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:32.305136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8db789-1c79-4015-966f-272dd1b0fbd4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.305568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8db789-1c79-4015-966f-272dd1b0fbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.306005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8db789-1c79-4015-966f-272dd1b0fbd4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.312974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 168/343] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:32.329618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.330129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.330576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.365601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:32.367428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:32.394250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:32.410391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18a8bf-6c1b-435b-92d0-4c1c3aa42525 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:32.473638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:32.474484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.474770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.475051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.498045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 169/344] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:32.500731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:32.509429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:32.510875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:32.511696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 176/345] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:32.520089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:32.539142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:32.570818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:32.708732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147a059-131a-4fec-ab09-8142be73ab97 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:32.759641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:32.760911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.761427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.762445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.775687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:32.786144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:32.787121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:32.787926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:32.790678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 177/346] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:32.791453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f7d9d-6200-4263-b766-5af6a1db33ec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:32.799191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:32.822712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:32.885477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:32.886478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.886760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.887064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.887896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:32.888281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:32.890648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859dead-c395-4116-8ea9-a89651020460 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:32.908401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 170/347] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:32.914547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99016-6ee3-4c37-a3d9-7ca859a6b836 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:32.914835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99016-6ee3-4c37-a3d9-7ca859a6b836 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:32.915224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99016-6ee3-4c37-a3d9-7ca859a6b836 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:32.918443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 171/348] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:32.922840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99016-6ee3-4c37-a3d9-7ca859a6b836 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:32.961371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4d23-8524-4932-a48e-aa1fe3c1a812 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.013843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.015153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.015643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.016376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.017770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:33.018309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.019917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1c99-4be5-453c-b4c2-99ccd639a17c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.033675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 178/349] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:33.035661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.036233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.036715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.059052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:33.060636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:33.082168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.098670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99016-6ee3-4c37-a3d9-7ca859a6b836 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.139968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.141148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.141444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.141799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.153490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 179/350] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:33.155025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.158814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.224925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.226566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.275816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa261a3a-a24b-4b23-8b8f-bbd535ef3d1e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.317190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.318220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.318705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.319329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.327946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.347093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 172/351] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:33.348157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.414429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.416879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.420381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.464823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d75cd-6b79-44ad-b91c-57a54f7fa800 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.519085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.520285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.520748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.521112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.535257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:33.538146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 180/352] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:33.539303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:33.544252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:33.545261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:33.546074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 173/353] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:33.546725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.607988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.664917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee69752-958d-405e-a440-fe5c31bce224 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.670739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.678679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0254e8-15e6-4aaf-a737-2d5410a850d9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.712924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.714311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.714806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.715309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.717119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.718913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.721356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 181/354] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:33.722642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.730056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.731032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.731406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.731752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.733106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.734777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.736798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 174/355] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:33.737734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.856371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86615fe5-2992-4f5f-9ffd-aa8c93dd7e63 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.897209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ddd18-449b-44a9-a24e-a7875073a0e1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:33.917374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.918076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.918358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.918627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.919679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:33.919992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.921406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2aea5-dbe5-45d4-bdcd-037600cca7cd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.926865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:33.938817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 182/356] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:33.940906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e5e87-24da-4d33-aa1a-6995f36e7932 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.941199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e5e87-24da-4d33-aa1a-6995f36e7932 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.941475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e5e87-24da-4d33-aa1a-6995f36e7932 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.943454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 183/357] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:33.944520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e5e87-24da-4d33-aa1a-6995f36e7932 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:33.953403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:33.954359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:33.954647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:33.954924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:33.964999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:33.966593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:33.968748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 175/358] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:33.969369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:34.008445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:34.108737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e5e87-24da-4d33-aa1a-6995f36e7932 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.137491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b344c-bb62-49d0-9365-6ceac22f0490 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.171572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d6ad1-f415-40e9-a069-3f3047b4ed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:34.172490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d6ad1-f415-40e9-a069-3f3047b4ed80 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.172774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d6ad1-f415-40e9-a069-3f3047b4ed80 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.173048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d6ad1-f415-40e9-a069-3f3047b4ed80 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.174261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d6ad1-f415-40e9-a069-3f3047b4ed80 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:34.175719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d6ad1-f415-40e9-a069-3f3047b4ed80 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:34.177477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 184/359] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:34.178812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.179103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.179380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.180388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acdadd-a61e-443c-adb2-53b6ce38de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:34.181082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acdadd-a61e-443c-adb2-53b6ce38de9b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.181351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acdadd-a61e-443c-adb2-53b6ce38de9b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.181659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acdadd-a61e-443c-adb2-53b6ce38de9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.199047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:34.200352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:34.250231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:34.274973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acdadd-a61e-443c-adb2-53b6ce38de9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:34.276828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acdadd-a61e-443c-adb2-53b6ce38de9b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:34.290052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 185/360] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:34.291292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:34.334674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:34.410798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 176/361] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:34.412416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429379-6839-4e9e-af32-a356745cfa3c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.412865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429379-6839-4e9e-af32-a356745cfa3c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.413326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429379-6839-4e9e-af32-a356745cfa3c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.415074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 177/362] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:34.415598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429379-6839-4e9e-af32-a356745cfa3c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:34.440207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b337d-446c-47e8-91a8-3261cf3b8e59 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.487765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:34.488858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.489301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.489581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.490441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:34.490755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:34.492286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:34.506284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 186/363] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:34 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:34.506718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:34.539471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:34.585630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429379-6839-4e9e-af32-a356745cfa3c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.644543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:34.645483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.645764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.646039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.646901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:34.647340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:34.648806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d6598-ffc8-486c-8543-b9fe62eabf6b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:34.657873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 178/364] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:34 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:34.659672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.660092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.660615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.669953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:34.689752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:34.691371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:34.695180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05e4a-e490-4ab9-8c74-9dc70520737d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.767723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:34.769081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.769377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.770541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.785076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:34.790458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 179/365] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:34.791954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:34.799101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:34.800284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:34.801164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:34.803301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 187/366] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:34.813521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:34.908530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:34.921021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b322af-1e65-4992-84e4-5723e0d55c18 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.954782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd82fb4-3879-4999-b629-54d67364618d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:34.963312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:34.964476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:34.964831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:34.965177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:34.966540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:34.968320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:34.970151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 180/367] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:34 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:34.971155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.004189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c810b-2915-444f-8216-c714e660e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.005272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c810b-2915-444f-8216-c714e660e66d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.006984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c810b-2915-444f-8216-c714e660e66d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.007255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c810b-2915-444f-8216-c714e660e66d None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.008026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c810b-2915-444f-8216-c714e660e66d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:35.008357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c810b-2915-444f-8216-c714e660e66d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:35.009867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c810b-2915-444f-8216-c714e660e66d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:35.028354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 188/368] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:35.031519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c810b-2915-444f-8216-c714e660e66d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.060823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:35.156954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc398-9157-4177-b215-5810e08c9ec2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.175561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c810b-2915-444f-8216-c714e660e66d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.206036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.206966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.207392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.207674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.212296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.213360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.213799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.214100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.215480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:35.218610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:35.219323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 189/369] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:35.220201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.222418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:35.231595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:35.232611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:35.233419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:35.236216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 181/370] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:35.237585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.243467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:35.334956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:35.352930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0aad0-3a0e-4997-8deb-c93275f3a1da None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.397754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.398431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.398719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.399008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.405595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e42045-b221-4a95-8fd8-d84889333413 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.446011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.446011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.446412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.446610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.448438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=8428242979624dc789b4621eb65f27a5) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:35.448848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:35.450340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:35.457699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 182/371] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:35.458653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.508567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:35.513389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:35.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:35.592913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54c5c8-c584-4075-8d1d-5590baecb102 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.644487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:35.665469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.666752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.667060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.667525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.669286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:35.669597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:35.671009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:35.675256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 190/372] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:35.681175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.690091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 183/373] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:35.696846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.855278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2247d-2114-41b3-8457-dacbb0130d59 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.887382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34b11b-867d-486c-86e2-ab4d3c62750d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:35.897407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.898297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.898579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.898855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.900196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:35.901804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:35.903972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 191/374] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:35.909742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:35.932404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:35.949992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:35.950599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:35.950883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:35.951154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:35.979182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:36.033385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c18b1-65e4-410d-8c2b-8087f0b139ff None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:36.079220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:36.080798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:36.105659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:36.106524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:36.106804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:36.107083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:36.108538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:36.110008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:36.111828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 192/375] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:36.120593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:36.243506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 184/376] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:36.249914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:36.292432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:36.322464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc9497-dbcb-4b4e-8514-93f94f493c78 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:36.349781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:36.367018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:36.367744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:36.368037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:36.368350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:36.370290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=8428242979624dc789b4621eb65f27a5) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:36.370625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:36.372444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:36.381046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 193/377] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:36 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:36.419707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d85438-70b8-4c8c-883c-7d7ab6ec1fb8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:36.461109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:36.461990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:36.462276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:36.463060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] PATH_INFO: `/v3/projects/243115c739b84006986ba2e6345f2509` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:36.464584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] RBAC: Authorizing `identity:get_project(project_id=243115c739b84006986ba2e6345f2509)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:36.466100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:36.467879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 185/378] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:36 2022] GET /identity/v3/projects/243115c739b84006986ba2e6345f2509 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:36.473570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:36.594896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fada83-4407-44a7-a274-d208b6ec138b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:36.634249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f839b1-dcfa-4dc3-a062-fc0ee6952d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:36.635824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f839b1-dcfa-4dc3-a062-fc0ee6952d45 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:36.636175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f839b1-dcfa-4dc3-a062-fc0ee6952d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:36.636413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f839b1-dcfa-4dc3-a062-fc0ee6952d45 None admin] PATH_INFO: `/v3/groups/e3960412d3df4f6ab0f69030d9e40bc4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:36.643649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f839b1-dcfa-4dc3-a062-fc0ee6952d45 None admin] RBAC: Authorizing `identity:get_group(group_id=e3960412d3df4f6ab0f69030d9e40bc4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:36.645214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f839b1-dcfa-4dc3-a062-fc0ee6952d45 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:36.647073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 194/379] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:36 2022] GET /identity/v3/groups/e3960412d3df4f6ab0f69030d9e40bc4 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:36.653658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:36.762520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:36.768727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec5c2e-00da-40fb-a959-256a241ba3a5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:36.809874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:36.810558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:36.810911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:36.811194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:36.812120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] RBAC: Adding query filter params (group.id=e3960412d3df4f6ab0f69030d9e40bc4, role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=243115c739b84006986ba2e6345f2509) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:36.812444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:36.813898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db789183-3bec-41c9-a836-3bfb6b5bec86 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:36.820542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 186/380] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:36 2022] GET /identity/v3/role_assignments?group.id=e3960412d3df4f6ab0f69030d9e40bc4&scope.project.id=243115c739b84006986ba2e6345f2509&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:37.316256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1d96-732d-48e2-b76e-546196f75391 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.316256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1d96-732d-48e2-b76e-546196f75391 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.316256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1d96-732d-48e2-b76e-546196f75391 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.317232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 195/381] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:37.324428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9d10c-d5b7-4aff-bf15-d81b84c8c859 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.325024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9d10c-d5b7-4aff-bf15-d81b84c8c859 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.325525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9d10c-d5b7-4aff-bf15-d81b84c8c859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.345172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-91f9d10c-d5b7-4aff-bf15-d81b84c8c859 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:37.346707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-91f9d10c-d5b7-4aff-bf15-d81b84c8c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:37.433299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 187/382] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:37.477761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1d96-732d-48e2-b76e-546196f75391 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:37.501190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1abc-bde5-4829-a7e6-6f35d59e8f00 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.501839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1abc-bde5-4829-a7e6-6f35d59e8f00 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.502397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1abc-bde5-4829-a7e6-6f35d59e8f00 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.504363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 188/383] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:37.517219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.517819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.518318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.539836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:37.542386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:37.574421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:37.624540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 189/384] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:37.647453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:37.673731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:37.682354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1d96-732d-48e2-b76e-546196f75391 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:37.737724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:37.738985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.739554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.740069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.757156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:37.762442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed352f-cf38-43a0-9c0e-9bebcbeae4fd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:37.766229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:37.767141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:37.767931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:37.774048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 196/385] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:37.781446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:37.804350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:37.805404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.805882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.806352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.818834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:37.828090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:37.829033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:37.829826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:37.836146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 190/386] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:37.838117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:37.928241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:37.951028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6bed-8aec-414e-8d31-6682590f38be None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:37.952547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa139f4e-9ece-49b8-a611-45081f60c0ef None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:37.963011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:37.996678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:37.997686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:37.998095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:37.998505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:37.999393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:37.999827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.001396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.010178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:38.011027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.011416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.011808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.015792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:38.015792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.015792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.016431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 191/387] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:38.018284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.037879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 197/388] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:38.043734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.234426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad699c7-410b-47d8-be2a-882eaa3d1d7d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:38.254383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f43a6-f2e2-4240-81ff-16e241317526 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:38.309729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:38.310611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.310896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.311184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.315698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.317204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.323500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 192/389] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:38 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:38.325472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.331201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:38.331884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.332202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.332487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.339460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.341121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.342966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 198/390] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:38 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:38.350573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:38.353083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.458977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:38.480793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5e74-a4ba-4dfa-8831-e61f0c856ca6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:38.528414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a07d5-130f-430e-b92b-5234f5484163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:38.529903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a07d5-130f-430e-b92b-5234f5484163 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.530372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a07d5-130f-430e-b92b-5234f5484163 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.530844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a07d5-130f-430e-b92b-5234f5484163 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.532865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a07d5-130f-430e-b92b-5234f5484163 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.534334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a07d5-130f-430e-b92b-5234f5484163 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.536495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 193/391] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:38 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:38.542659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb85328-9700-4c34-8170-b3b501275bee None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.542999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb85328-9700-4c34-8170-b3b501275bee None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.543215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb85328-9700-4c34-8170-b3b501275bee None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.544921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 194/392] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:38.545549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb85328-9700-4c34-8170-b3b501275bee None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.552363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c86ef5-5185-46dc-8870-de4741f5143d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:38.597614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f9339-57b6-4938-88d5-6ced37d3819c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:38.598863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f9339-57b6-4938-88d5-6ced37d3819c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.599345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f9339-57b6-4938-88d5-6ced37d3819c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.599838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f9339-57b6-4938-88d5-6ced37d3819c None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.601624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f9339-57b6-4938-88d5-6ced37d3819c None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.604275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f9339-57b6-4938-88d5-6ced37d3819c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.611538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 199/393] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:38 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:38.613502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.614020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.614493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.637550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:38.639126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:38.656629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:38.688196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb85328-9700-4c34-8170-b3b501275bee None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:38.727486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:38.729549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:38.730059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:38.731125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/groups/4656be92c7ce412495392c74fdd47978/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:38.734141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] RBAC: Authorizing `identity:create_grant(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d, group_id=4656be92c7ce412495392c74fdd47978, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:38.735850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:38.736179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 200/394] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:38.737101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.754418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 195/395] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Jun 16 18:14:38 2022] PUT /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/groups/4656be92c7ce412495392c74fdd47978/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:38.772143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:38.836501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:38.977750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9be4b-fa04-4f78-a054-b94eddb473de None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:38.988037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2549be8-73c1-44ae-8765-37d06da7ba23 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:39.047614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:39.048495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.048764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.049043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/groups/4656be92c7ce412495392c74fdd47978/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.054043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d, group_id=4656be92c7ce412495392c74fdd47978, role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:39.058951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:39.066712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:39.067819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.068153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.068492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.081386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:39.083799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 201/396] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Jun 16 18:14:38 2022] PUT /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d/groups/4656be92c7ce412495392c74fdd47978/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:39.085417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:39.094824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:39.095833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:39.096688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:39.104255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 196/397] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:39.109823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:39.249161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:39.254616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82bff3-6126-4a75-9737-00091061ebb5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:39.303744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:39.304858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.305361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.306108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.308475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:39.309090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:39.311440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93e3a-31ff-4f5f-9c36-9d66d0a8543a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:39.330783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 202/398] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:39.338415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:39.509360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791cfe-df77-4e5a-b09d-8dbe7de06995 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:39.581239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda8e45-8ec9-437e-a667-2ae41ebeb297 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.581405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2eefc-1cd7-4ede-9c59-8ddd3bfedae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:39.581968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda8e45-8ec9-437e-a667-2ae41ebeb297 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.582086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2eefc-1cd7-4ede-9c59-8ddd3bfedae6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.582366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2eefc-1cd7-4ede-9c59-8ddd3bfedae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.582643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2eefc-1cd7-4ede-9c59-8ddd3bfedae6 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.582980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda8e45-8ec9-437e-a667-2ae41ebeb297 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.583858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2eefc-1cd7-4ede-9c59-8ddd3bfedae6 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:39.584834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 203/399] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:39.587453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2eefc-1cd7-4ede-9c59-8ddd3bfedae6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:39.589302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 197/400] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:39 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:39.593903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda8e45-8ec9-437e-a667-2ae41ebeb297 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:39.607498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f6a27-fe73-49d6-b490-b854538da19f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.607839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f6a27-fe73-49d6-b490-b854538da19f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.608071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f6a27-fe73-49d6-b490-b854538da19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.625777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d07f6a27-fe73-49d6-b490-b854538da19f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:39.626998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d07f6a27-fe73-49d6-b490-b854538da19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:39.799298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:39.817477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 198/401] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:39.821672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd30b-636e-4f60-8277-1c8ec040492a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.821672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd30b-636e-4f60-8277-1c8ec040492a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.821672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd30b-636e-4f60-8277-1c8ec040492a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.821672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 199/402] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:39.827978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.828301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.828583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.848157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:39.849441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:39.856695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda8e45-8ec9-437e-a667-2ae41ebeb297 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:39.898752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:39.899597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:39.899884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:39.900729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:39.902552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:39.905154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:39.911834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 204/403] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:39 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:39.911834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:39.949934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:39.953544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 200/404] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:39.960244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.059219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c75cd-3dae-4705-a98c-e8b57c4260f0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.078938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3384-3574-43c2-93a1-aa5eb6e6293d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.100343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.101839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.102299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.102720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.112796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:40.118664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:40.126182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.127304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.127709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.128126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5/groups/4656be92c7ce412495392c74fdd47978/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:40.129730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:40.130796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:40.131784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:40.132836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8428242979624dc789b4621eb65f27a5, group_id=4656be92c7ce412495392c74fdd47978, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:40.135063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 205/405] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:40.136500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.140019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:40.159932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 201/406] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Jun 16 18:14:39 2022] PUT /identity/v3/projects/8428242979624dc789b4621eb65f27a5/groups/4656be92c7ce412495392c74fdd47978/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:40.166548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.249491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:40.265085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce5444-9818-4c25-8d8d-3fe0c700b222 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.330726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.332237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.333862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.334350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.370519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:40.389507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:40.390872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:40.392225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:40.395986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 206/407] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:40.400126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1feb-cc4f-409b-b5d2-b9936a0eb5e4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.403050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.459382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.460140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.460428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.460711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.461604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:40.461984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:40.467237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:40.477900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 202/408] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:40.483559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.527144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966ed48-5646-4d65-815d-367104a68a17 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.560457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:40.572502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.573705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.574128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.574589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.575483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:40.575958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:40.577586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:40.591962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 207/409] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:40.593557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.607785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:40.614265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5be82-4e63-4097-8886-eb31d4310047 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.655339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.656413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.656871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.657276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] PATH_INFO: `/v3/groups/e3960412d3df4f6ab0f69030d9e40bc4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.658858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] RBAC: Authorizing `identity:get_group(group_id=e3960412d3df4f6ab0f69030d9e40bc4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:40.660587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:40.662878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 203/410] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/groups/e3960412d3df4f6ab0f69030d9e40bc4 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:40.668957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.730892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60161-14f6-400c-a297-8a2eca717edb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.776886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.778149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.778625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.779110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.781687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:40.783345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:40.785251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 208/411] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:40.793388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.877610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce483f17-07d4-4bc4-ab92-503cf045bb62 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:40.903668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:40.930231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:40.932871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:40.933384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:40.934224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] PATH_INFO: `/v3/projects/243115c739b84006986ba2e6345f2509` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:40.935984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] RBAC: Authorizing `identity:get_project(project_id=243115c739b84006986ba2e6345f2509)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:40.938522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:40.942519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 204/412] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/projects/243115c739b84006986ba2e6345f2509 => generated 344 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:40.951212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:40.967244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:41.015183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a2c8e-6788-4024-98cd-ac24c9d6c2b8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:41.083424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:41.086588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.086869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.087143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.089612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:41.092220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:41.095904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 209/413] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:40 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:41.099555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:41.160463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dbd61-d651-4f3b-8c5c-55130bc2d07a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:41.241351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652dcf9-a113-4320-863d-4e5cf3b7f944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:41.242180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652dcf9-a113-4320-863d-4e5cf3b7f944 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.242461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652dcf9-a113-4320-863d-4e5cf3b7f944 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.242732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652dcf9-a113-4320-863d-4e5cf3b7f944 None admin] PATH_INFO: `/v3/projects/243115c739b84006986ba2e6345f2509/groups/e3960412d3df4f6ab0f69030d9e40bc4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.248460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652dcf9-a113-4320-863d-4e5cf3b7f944 None admin] RBAC: Authorizing `identity:create_grant(project_id=243115c739b84006986ba2e6345f2509, group_id=e3960412d3df4f6ab0f69030d9e40bc4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:41.249932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652dcf9-a113-4320-863d-4e5cf3b7f944 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:41.278696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 205/414] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Jun 16 18:14:40 2022] PUT /identity/v3/projects/243115c739b84006986ba2e6345f2509/groups/e3960412d3df4f6ab0f69030d9e40bc4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:41.289837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52c0e-f6d0-4c79-a438-5fa79dbfdaf9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:41.354639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59a77e-298f-4ec0-8add-5e5c8a0d6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:41.356148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a77e-298f-4ec0-8add-5e5c8a0d6e13 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.356651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a77e-298f-4ec0-8add-5e5c8a0d6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.357158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a77e-298f-4ec0-8add-5e5c8a0d6e13 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5/groups/4656be92c7ce412495392c74fdd47978/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.361173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a77e-298f-4ec0-8add-5e5c8a0d6e13 None admin] RBAC: Authorizing `identity:create_grant(project_id=8428242979624dc789b4621eb65f27a5, group_id=4656be92c7ce412495392c74fdd47978, role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:41.363891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a77e-298f-4ec0-8add-5e5c8a0d6e13 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:41.389882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 210/415] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Jun 16 18:14:41 2022] PUT /identity/v3/projects/8428242979624dc789b4621eb65f27a5/groups/4656be92c7ce412495392c74fdd47978/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:14:41.853676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b36d-364e-4d05-941b-a5cc72e5914e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.853676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b36d-364e-4d05-941b-a5cc72e5914e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.854599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b36d-364e-4d05-941b-a5cc72e5914e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.857409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 206/416] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:41.863935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.864493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.864976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.879759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:41.891102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:41.892353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:41.945133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578adb9-a1e4-4a3b-8f7c-7bf45781d28a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.946084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578adb9-a1e4-4a3b-8f7c-7bf45781d28a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.946608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578adb9-a1e4-4a3b-8f7c-7bf45781d28a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.948568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 207/417] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:41.954067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c70821-bc14-4816-9206-279178f38f04 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:41.954365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c70821-bc14-4816-9206-279178f38f04 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:41.954656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c70821-bc14-4816-9206-279178f38f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:41.985635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-09c70821-bc14-4816-9206-279178f38f04 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:41.986863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-09c70821-bc14-4816-9206-279178f38f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:41.990638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 211/418] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:42.038462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:42.058019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:42.087839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 208/419] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:42.123372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c70821-bc14-4816-9206-279178f38f04 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:42.203262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1dae9-9a6b-4530-b0a6-a53ec0a1a19f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:42.268913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:42.269742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.270029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.270305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.276872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:42.291283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:42.299460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:42.300427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:42.301220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 212/420] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:42.307454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:42.338596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c70821-bc14-4816-9206-279178f38f04 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:42.425210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec0421-990d-49d8-a02a-8864d6fbff40 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:42.450167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:42.450992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.451277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.451565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.463543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:42.464255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.464541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.464817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.465586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:42.465900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:42.467291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:42.476797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 213/421] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:42.482910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:42.484702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:42.493073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:42.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:42.497735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:42.503848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 209/422] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:42.509110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:42.550747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:42.596553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87d35-7f0c-48d2-8c30-50d820cc8a77 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:42.623544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:42.634604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3fc9-d76b-4355-9cf8-cc71c753ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:42.635764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3fc9-d76b-4355-9cf8-cc71c753ddbc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.636063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3fc9-d76b-4355-9cf8-cc71c753ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.636386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3fc9-d76b-4355-9cf8-cc71c753ddbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.753025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3fc9-d76b-4355-9cf8-cc71c753ddbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:42.755517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3fc9-d76b-4355-9cf8-cc71c753ddbc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:42.796795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941d0cc-f066-48df-9edb-6a2e4a64e4c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:42.855618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:42.856335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.856616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.856890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.857651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:42.858025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:42.859395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad326016-745b-49d6-be67-8ca55d9edd6f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:42.869743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 210/423] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:42.872030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b946e-6108-4af6-828d-655f09c41662 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.872030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b946e-6108-4af6-828d-655f09c41662 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.872030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b946e-6108-4af6-828d-655f09c41662 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.872926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 211/424] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:42.877585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b946e-6108-4af6-828d-655f09c41662 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:42.882304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:42.956418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 214/425] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:42.958772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:42.959339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:42.959815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:42.989344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:42.990892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:43.017587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.029324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b946e-6108-4af6-828d-655f09c41662 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.074803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.075480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.075761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.076580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.094901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 215/426] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:43.096330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.155106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.157346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.199541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.215639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628738a-d559-422f-a130-01ae04a8ec97 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.259117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.261103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.261748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.262328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.263480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.265487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.267245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.269488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 216/427] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:43.270631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.274937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 212/428] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:43.276613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.382526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c978d-2875-4ef7-aefe-2b0626a91fd1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.390258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c3d8c-828b-4bd1-be25-86a5c5f94b91 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.420729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.421737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.422087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.422427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.423722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.425290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.427135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 213/429] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:43.432458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.432739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.434039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.434528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.435022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.448507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:43.457210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:43.458179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:14:43.458986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:43.461568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 217/430] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:43.463708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.486045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.499379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.548781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354bfec-7ea9-41e7-a07d-68312ecf3b3e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.585910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.586954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.587279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.587600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] PATH_INFO: `/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.589235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] RBAC: Authorizing `identity:get_project(project_id=659d7d2e6b2d4e0484e8acd3e74c8b0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.590705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.592847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 214/431] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/projects/659d7d2e6b2d4e0484e8acd3e74c8b0d => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:43.593689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.598141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4867fa-ece1-4de1-a15f-5e05ca74c7a9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.647499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.649009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.649757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.650538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.651723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:43.652342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.654082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.664611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 218/432] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:43.666748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.768207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.792815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317fcae-336c-4e50-9f01-56f03184c827 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.814463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e72ac-e71e-4a7a-88d4-18d550d0a74c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:43.833926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.835226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.835714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.836239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.837895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.840140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.842491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 219/433] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:43.849198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:43.894265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:43.901262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:43.902053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:43.902338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:43.902683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:43.903483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:43.903792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:43.905726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:43.919032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 215/434] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:43.929308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.002204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc4bc4-4d80-4767-b39c-3ea9aa3ddd84 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.048597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.049290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.049725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.050118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.050977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:44.051295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:44.053383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7ab6-6702-41ab-8842-aea94130da1e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:44.063228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 220/435] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=659d7d2e6b2d4e0484e8acd3e74c8b0d&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:44.115462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0cb3-01c8-4581-8fed-16bbda6369bc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.159948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.161236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.161759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.162185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.174778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee27f3c-3672-4a7a-8e26-03178d461fc9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.175219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee27f3c-3672-4a7a-8e26-03178d461fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.175601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee27f3c-3672-4a7a-8e26-03178d461fc9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.177463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 221/436] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:44.184356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.184780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.185167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.204014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:44.205431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:44.214194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:44.238411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:44.248564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:44.250058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:44.289761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 222/437] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:44.318342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.375610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 216/438] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:44.382029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.462042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0502ad0-b152-41b8-bcb9-2f356397fe54 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.493433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:44.511462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98378120-5cca-4054-8260-e3ab58c6770e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.515968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bc74a-9049-4e5d-bade-4e83b8bbbae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.516861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bc74a-9049-4e5d-bade-4e83b8bbbae0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.517146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bc74a-9049-4e5d-bade-4e83b8bbbae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.517420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bc74a-9049-4e5d-bade-4e83b8bbbae0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.534966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bc74a-9049-4e5d-bade-4e83b8bbbae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-375bc74a-9049-4e5d-bade-4e83b8bbbae0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:44.544486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:44.545409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:44.546207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:44.551469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 223/439] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:44.552949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de81596-0677-4220-a15d-a740782e2277 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.553237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de81596-0677-4220-a15d-a740782e2277 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.553516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de81596-0677-4220-a15d-a740782e2277 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.554856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 224/440] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:44.555548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de81596-0677-4220-a15d-a740782e2277 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.559239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1fdeb-1c2f-415a-b197-511b74a06d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.560150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:44.560294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1fdeb-1c2f-415a-b197-511b74a06d2e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.560531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1fdeb-1c2f-415a-b197-511b74a06d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.561666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1fdeb-1c2f-415a-b197-511b74a06d2e None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.564241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1fdeb-1c2f-415a-b197-511b74a06d2e None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:44.564241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1fdeb-1c2f-415a-b197-511b74a06d2e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:44.565957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 217/441] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:44.567015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.567296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.567945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.591673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:44.593023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:44.664218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 218/442] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:44.664981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.674962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de81596-0677-4220-a15d-a740782e2277 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.712097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.713338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.713642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.713917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.714680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:44.714985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:44.716411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:44.724522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 225/443] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:44.725328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.760272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:44.770959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07baf-a4b8-4a84-8b97-bd07199d7910 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.793157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:44.809411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.810394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.810747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.811088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.812385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:44.813841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:44.815468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 219/444] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:44.816258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.836579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47e6c4-1c46-4c64-a872-d27e2bb646b7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.875453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.876281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.876566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.876846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.887791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:44.895551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:44.896458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:44.897475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 226/445] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:44.898299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:44.926606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840ac3-dc09-4d41-9651-16e5a665eb7e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:44.964704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:44.965389 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:44.965674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:44.965952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:44.998748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.009265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c3224-3636-42ba-b726-db61e9d2a6b9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.037247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.045858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.047349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.050880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.051556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.051837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.052144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.052995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=f21b9c85de034e5fac7761764c96d2f6, scope.project.id=8428242979624dc789b4621eb65f27a5) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:45.053308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.054777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.062707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 227/446] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=f21b9c85de034e5fac7761764c96d2f6 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:45.063579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.179325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 220/447] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:45.185795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.211374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0be22-3edc-4849-89df-e36e69b937e0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.256131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.256841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.257278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.257571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.258159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.258719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:45.259570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.261509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.271909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 228/448] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:45.280444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.311999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e49ee-1edd-4618-8fcc-649e3e91ce5d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.347055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.365694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.366702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.367620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.367620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] PATH_INFO: `/v3/projects/8428242979624dc789b4621eb65f27a5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.370100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] RBAC: Authorizing `identity:get_project(project_id=8428242979624dc789b4621eb65f27a5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.370100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.372020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 221/449] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/projects/8428242979624dc789b4621eb65f27a5 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:45.377408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.420789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26611b4c-a670-4dca-ac42-b5f6823a7421 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.467777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.470307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.470744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.474363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.521584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242371b-fca1-46f2-822e-c2790132cde0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.557081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.559027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.565623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.577310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.579283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.579762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.580271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] PATH_INFO: `/v3/groups/4656be92c7ce412495392c74fdd47978` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.581559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] RBAC: Authorizing `identity:get_group(group_id=4656be92c7ce412495392c74fdd47978)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.583007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.584853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 222/450] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/groups/4656be92c7ce412495392c74fdd47978 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:45.591473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.622850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.680301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 229/451] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:14:45.686519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.710357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb8dc-0637-4a55-87bb-f01edb92a488 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.749259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.750264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.750698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.751128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.752198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] RBAC: Adding query filter params (group.id=4656be92c7ce412495392c74fdd47978, role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=8428242979624dc789b4621eb65f27a5) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:45.752768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.754393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.761085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 223/452] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/role_assignments?group.id=4656be92c7ce412495392c74fdd47978&scope.project.id=8428242979624dc789b4621eb65f27a5&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:45.809555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910933-c980-4670-a674-d1a9d6c2aa77 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:45.847646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:45.848582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:45.848863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:45.849135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] PATH_INFO: `/v3/projects/243115c739b84006986ba2e6345f2509` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:45.850416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] RBAC: Authorizing `identity:get_project(project_id=243115c739b84006986ba2e6345f2509)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:45.851886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:45.854382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 230/453] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/projects/243115c739b84006986ba2e6345f2509 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:14:45.860063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:45.924062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:45.968111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ca280-2369-4b3f-966e-5cbae54b920a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:46.002072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadc6c2-d41e-4815-9f9e-fce386499443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:46.002872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc6c2-d41e-4815-9f9e-fce386499443 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:46.003161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc6c2-d41e-4815-9f9e-fce386499443 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:46.003441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc6c2-d41e-4815-9f9e-fce386499443 None admin] PATH_INFO: `/v3/groups/e3960412d3df4f6ab0f69030d9e40bc4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:46.004636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadc6c2-d41e-4815-9f9e-fce386499443 None admin] RBAC: Authorizing `identity:get_group(group_id=e3960412d3df4f6ab0f69030d9e40bc4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:46.006071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadc6c2-d41e-4815-9f9e-fce386499443 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:46.007680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 224/454] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Jun 16 18:14:45 2022] GET /identity/v3/groups/e3960412d3df4f6ab0f69030d9e40bc4 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:46.014228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:46.035334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:46.123520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5282b2-309f-4e59-be6e-17ccee10dc38 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:46.157994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:46.158649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:46.158934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:46.159214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:46.160114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] RBAC: Adding query filter params (group.id=e3960412d3df4f6ab0f69030d9e40bc4, role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=243115c739b84006986ba2e6345f2509) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:46.160431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:46.161855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20e96c-a32c-49af-aeb7-b1ff1c960f77 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:46.167758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 231/455] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Jun 16 18:14:46 2022] GET /identity/v3/role_assignments?group.id=e3960412d3df4f6ab0f69030d9e40bc4&scope.project.id=243115c739b84006986ba2e6345f2509&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:48.887373 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6e30-fbf7-4a37-9e4b-19bb5969e9ae None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:48.889144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6e30-fbf7-4a37-9e4b-19bb5969e9ae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:48.889955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6e30-fbf7-4a37-9e4b-19bb5969e9ae None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:48.893503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 225/456] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:48.903050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cbf40-0fe0-42d1-934d-6a9ffd66904b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:48.904427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cbf40-0fe0-42d1-934d-6a9ffd66904b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:48.905422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cbf40-0fe0-42d1-934d-6a9ffd66904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:48.936641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2a6cbf40-0fe0-42d1-934d-6a9ffd66904b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:48.939156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2a6cbf40-0fe0-42d1-934d-6a9ffd66904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:49.072712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 232/457] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:49.108898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6e30-fbf7-4a37-9e4b-19bb5969e9ae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:49.311817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9697-28a1-49b2-b5be-ce7f51dde1a1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:49.312772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9697-28a1-49b2-b5be-ce7f51dde1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:49.313504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9697-28a1-49b2-b5be-ce7f51dde1a1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:49.316948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 233/458] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:14:49.331567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:49.332068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6e30-fbf7-4a37-9e4b-19bb5969e9ae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:49.332811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:49.333527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:49.366456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:49.368961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:49.417583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:49.420237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:49.451317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:49.457830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:49.458762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:49.459523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:49.510019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:49.519161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:49.560161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 234/459] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:49.581910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:49.583793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:49.585732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:49.585732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:49.585732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:49.585732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:49.585732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 226/460] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:49.593778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:49.628011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:49.831618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a94a56-9064-4074-8fba-41e9179083d8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:49.877964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:49.878797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:49.879082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:49.879345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:49.908345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:49.909372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662604e-b4ae-434c-948a-82ee03210e32 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:49.913298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-645542d6-28ff-4afa-8cf5-4ba0daedd334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:49.941717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:49.943574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 227/461] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687153a-f79a-4c80-a150-f530e97ca0c3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:49.945371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687153a-f79a-4c80-a150-f530e97ca0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:49.946072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687153a-f79a-4c80-a150-f530e97ca0c3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:49.946828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 228/462] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:49.952170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687153a-f79a-4c80-a150-f530e97ca0c3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:49.977300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:49.979478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:49.980293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:49.980928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:49.982460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:49.983201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:49.985979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf0aca-6090-428a-8706-cabfa9d3199c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:50.007390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 235/463] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:50.009966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.010674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.011320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.016635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:50.020614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:50.042792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:50.045103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:50.152003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687153a-f79a-4c80-a150-f530e97ca0c3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:50.208557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 236/464] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:50.210101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:50.211560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:50.212959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.213656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.214291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.215778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:50.216474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:50.219003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:50.238047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 229/465] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:50.258482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:50.384014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:50.395690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a69e2f-c273-4344-95f2-a6b12d6b8e56 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:50.426819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c058e2-7793-47e0-93c0-1a2607417557 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:50.440871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:50.441965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.442443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.442800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.443526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:50.445480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:50.468574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:50.475029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 230/466] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:50 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:14:50.475953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:50.485723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24a8ea-d893-45d2-b4af-aeb4bf05c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:50.487225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24a8ea-d893-45d2-b4af-aeb4bf05c392 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.487927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24a8ea-d893-45d2-b4af-aeb4bf05c392 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.488877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24a8ea-d893-45d2-b4af-aeb4bf05c392 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.491479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24a8ea-d893-45d2-b4af-aeb4bf05c392 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:50.496991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24a8ea-d893-45d2-b4af-aeb4bf05c392 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:50.541412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 237/467] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:50 2022] POST /identity/v3/users => generated 246 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:50.541412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764d32-5557-479e-8009-02ccde82cbd4 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.541412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764d32-5557-479e-8009-02ccde82cbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.541412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764d32-5557-479e-8009-02ccde82cbd4 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.553502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 238/468] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:50.572772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.574043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.574680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.638423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:50.640880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:50.717419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6947-0272-4005-9cce-57fd80fccff9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:50.768420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 239/469] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:50.791568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:50.792452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:50.792737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:50.793016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:50.812400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:50.814090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:50.824832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:50.828196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:50.843854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:50.846261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:50.848417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:50.848417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:50.848417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:50.848417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:50.848417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 231/470] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:50.856355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:51.035123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6266-98e5-460f-bc77-47556c418ae8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:51.070076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:51.073030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fce9c8-4a75-4ddb-87bd-f1c428dafb62 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:51.102889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:51.104797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.105236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.105704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.125955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:51.128800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:51.142093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:51.143736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.144571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.145396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.146694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:51.147416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:51.151439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:51.166851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:51.169062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:51.171288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:51.171288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:51.171288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:51.171288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:51.176696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 240/471] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:14:51.180252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:51.183909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 232/472] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:51.193789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:51.307041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:51.334468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdb665-178b-48ac-9973-c3179f81c116 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:51.382170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ddffd-963a-48a7-88b6-6cb2f2605c87 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:51.385215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:51.386346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.386878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.387169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.389004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:51.389357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:51.390820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114c4f-ae2a-4446-ad48-0ef6ae9966da None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:51.407061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 241/473] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:51.409246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7125ff-f60a-46e4-890c-c701596f16e1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.409895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7125ff-f60a-46e4-890c-c701596f16e1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.410551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7125ff-f60a-46e4-890c-c701596f16e1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.415931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 242/474] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:51.421221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7125ff-f60a-46e4-890c-c701596f16e1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:51.461657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002f4b-00c1-4bd8-abdf-c2dd97e33395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:51.462790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002f4b-00c1-4bd8-abdf-c2dd97e33395 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.463389 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002f4b-00c1-4bd8-abdf-c2dd97e33395 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.463779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002f4b-00c1-4bd8-abdf-c2dd97e33395 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.465235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82002f4b-00c1-4bd8-abdf-c2dd97e33395 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:51.468035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82002f4b-00c1-4bd8-abdf-c2dd97e33395 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:51.490005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:51.512252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 233/475] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:51 2022] POST /identity/v3/users => generated 248 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:51.515229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.520228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.521087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.548448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:51.550453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:51.590712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7125ff-f60a-46e4-890c-c701596f16e1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:51.635704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:51.636800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.637290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.637694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.638582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:51.640417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:51.676679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 243/476] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:51 2022] POST /identity/v3/users => generated 249 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:51.683587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 234/477] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:51.734279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:51.867940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:51.888365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02839413-9952-4ab7-b40c-2784c9464253 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:51.937144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:51.937951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:51.938257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:51.938620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:51.939354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:51.940882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:51.953406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.sql.core [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:14:51.953406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 18:14:51.953406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [parameters: {'id': '5b8364eba68544b6b97cb97a958a8f59', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 18:14:51.953406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80147) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result.read() Jun 16 18:14:51.973728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:14:51.975209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application yield resource Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise self.value Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application session.commit() Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:14:51.979017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:14:51.989365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:14:51.992540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result.read() Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:14:51.999874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [parameters: {'id': '5b8364eba68544b6b97cb97a958a8f59', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:52.003879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 18:14:52.007159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:52.014426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 244/478] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Jun 16 18:14:51 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:52.014883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:52.068532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:52.155925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7129286-2fb8-45a0-ab0b-878b6ef8b599 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:52.211292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5bbdf-75a5-42f0-ac0f-f100e35efb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:52.212802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5bbdf-75a5-42f0-ac0f-f100e35efb08 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:52.213328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5bbdf-75a5-42f0-ac0f-f100e35efb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:52.213841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5bbdf-75a5-42f0-ac0f-f100e35efb08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:52.226906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5bbdf-75a5-42f0-ac0f-f100e35efb08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-48f5bbdf-75a5-42f0-ac0f-f100e35efb08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:52.236477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:52.237552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:14:52.238442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:52.241412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 235/479] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:14:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:52.251563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:52.398811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99da194-bb21-4d4f-82aa-4d27d19fe3b8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:52.448783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:52.449485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:52.449778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:52.450138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:52.451006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:52.451327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:52.452917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773af77-afe2-4975-8db9-db9ebc57e2e8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:52.463769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 245/480] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:54.616670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84911f0-7dc6-4084-81bb-fd53994cda71 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:54.617692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84911f0-7dc6-4084-81bb-fd53994cda71 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:54.617692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84911f0-7dc6-4084-81bb-fd53994cda71 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:54.619166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 236/481] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:54.625501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb339c-30c0-4014-ae5f-afbe552baf8b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:54.626266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb339c-30c0-4014-ae5f-afbe552baf8b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:54.626266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb339c-30c0-4014-ae5f-afbe552baf8b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:54.627853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 246/482] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:54.629357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad400abe-0e84-4aef-b08c-6031f0ed803e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:54.629675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad400abe-0e84-4aef-b08c-6031f0ed803e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:54.629983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad400abe-0e84-4aef-b08c-6031f0ed803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:54.651461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ad400abe-0e84-4aef-b08c-6031f0ed803e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:54.652894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ad400abe-0e84-4aef-b08c-6031f0ed803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:54.653925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:54.654277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:54.654500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:54.679418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:54.681089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:54.692294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:54.737561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 237/483] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:54.739185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a296257-7f3a-435a-a6c9-e2d7ae0e00c1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:54.739623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a296257-7f3a-435a-a6c9-e2d7ae0e00c1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:54.740065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a296257-7f3a-435a-a6c9-e2d7ae0e00c1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:54.741985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 238/484] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:54.750004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:54.750684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:54.751114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:54.774570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:54.776648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:54.819706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:54.824279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 247/485] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:54.827372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:54.884183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 239/486] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:14:54.886176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.033676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2e397-80ed-4e2c-8b5b-53442420b2da None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.051823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32627d56-9286-4d2f-a7d5-2422125039f9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.073977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.075174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.075640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.076137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.089122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:55.097730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:55.098691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:55.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:55.108116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 240/487] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:14:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:55.109522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.136203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.137989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.138628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.139391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.149786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:55.169680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:55.184839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:55.185807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:55.186607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:55.187235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 248/488] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:14:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:55.189037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.193710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:55.258175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcaaf41-3cb6-414c-996c-800abb414b6d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.323582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.324355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.324678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.324917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.346510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:55.363297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:55.365297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:55.367253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 241/489] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:55.370175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.382373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe93b3-8db9-479b-92a4-c6782917cd28 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.443701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.444483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.444973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.445499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.446831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:55.447923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:55.449755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc400033-8dc1-4273-992b-34f52cf8b4d4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:55.461122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 249/490] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:55.463151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d55974-5799-47c9-b91d-09def19dc3a7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.464213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d55974-5799-47c9-b91d-09def19dc3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.464497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d55974-5799-47c9-b91d-09def19dc3a7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.470001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 250/491] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:55.471483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d55974-5799-47c9-b91d-09def19dc3a7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.526503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:55.546421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50f24a-3f9c-465f-b8a6-3d0290dfb1a1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.587565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.588849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.589320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.589599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:55.589723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.590394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:55.590778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:55.592354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:55.607958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 242/492] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:55.607958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.645140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d55974-5799-47c9-b91d-09def19dc3a7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.690883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.691895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.692432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.692942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.694322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:55.694864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:55.698565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f438b3d-4a5e-49b3-93a1-5023e9c37f89 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:55.709981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 251/493] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:55.711472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.711883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.712233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.732752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:55.733976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:55.773983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9aeb0-aeb2-4e87-a0b4-a4ba86e99ebf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:55.830927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 252/494] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:55.831724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.842375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:55.845469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:55.847298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:55.848191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:55.872283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:55.875743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:55.892498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:55.908790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:55.914025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:55.915214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:55.916785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:55.916785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:55.916785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:55.916785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:55.918727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 243/495] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:55.919588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:55.977243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56d059-ca00-4bcd-97ee-07387d2f6023 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.022079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.022901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.023182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.023455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.036458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.038132 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.043781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811c8d0-fdc3-4b71-afcf-c934ef15af94 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:56.059868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:56.061089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:56.062047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:56.062047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:56.062047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:56.062047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:56.062047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 253/496] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:56.062324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.088774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.089956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.090266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.090825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.110247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.113172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4c3112e3-f43f-424b-9256-19232b0b58f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:56.136812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:56.137826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:56.138705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:56.138705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:56.138705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:56.138705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:56.138705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 244/497] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:56.139423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740765-bd90-4bc0-85bd-23aa7de49854 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.139765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740765-bd90-4bc0-85bd-23aa7de49854 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.140077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740765-bd90-4bc0-85bd-23aa7de49854 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.141496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 245/498] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:14:56.142400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740765-bd90-4bc0-85bd-23aa7de49854 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.223057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:56.226581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59303da7-a836-41eb-b698-58c235bad338 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.240503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:56.287953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740765-bd90-4bc0-85bd-23aa7de49854 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.288255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.289542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.290050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.290722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.307301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:14:56.316039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:56.317057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:56.317855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 254/499] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:56.318675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.344388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.345349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.345821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.346230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.347249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:56.347768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.349423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.367289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 246/500] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:56.368154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.466027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bcc50-a239-4659-ad6a-d7167eb1db56 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.531925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.533021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.533479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.533934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.535207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:56.535661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.538504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215eb30d-6e68-4558-b49d-72eff93173cc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.540137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a4311-fe2d-47e6-985b-d3fa81a2171d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.560668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 255/501] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:56.562534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.563113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.563587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.566378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:56.591327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:56.593422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:56.597234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.598030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.598349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.598637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.599541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:56.599852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.601731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.615913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 247/502] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:56.617606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.635263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:56.689460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 256/503] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:14:56.692089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.771160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3250d4-3839-4bdd-91ab-b6b38c31c9b8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.817321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.818136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.818377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.818662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.819506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:56.819970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.821671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.836202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 248/504] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:56.836810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.857704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21ab82-cce3-46f3-ac16-04f6f31b1a48 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:56.897029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:56.897637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:56.897804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:56.898077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:56.898236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:56.909384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:56.911596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:14:56.933053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:56.939949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 257/505] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:56.941428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:56.962344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.004481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26047658-2e37-4c46-9e30-23f6423a7bb2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.068120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.069063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.069347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.069625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.079997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.080316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a6a06-9ddc-41df-ad7b-65d2bcebca83 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.081513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:14:57.099718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:57.101046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 249/506] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:57.102848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.137460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.138783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.139146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.139553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.152685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.155573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:14:57.175258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:57.176331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:57.177256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 258/507] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:14:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:57.177806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.235755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.248338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825832-b6c5-40d9-8234-ef8e0df1bf75 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.292000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.295752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.296806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.297123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.297526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.312468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.314061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.320812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a97b37-a57f-4596-9637-6368af94cb86 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:57.337575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:57.339299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:57.341696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:57.341696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:57.341696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:57.341696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:57.341696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 250/508] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:57.341696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.383842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.385452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.385951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.386381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.401479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.403459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:57.425903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:57.427285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:57.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:57.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:57.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:57.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:57.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 259/509] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:57.429281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.471726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e24ed-5779-4939-bf90-b0558615981d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.514434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.514434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.514723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.515090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.516738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:57.517231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.519724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.564084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 251/510] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:57.565498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.571950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.586500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a1ef2-0bcc-44c7-9749-477e89a98a1c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.622933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.630333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.631193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.631484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.631908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.633378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:57.633776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.636262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.681394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 260/511] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:57.682988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.709640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a2491-13bc-434d-ad00-57ff4005dd8d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.751737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.752559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.752850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.753126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.754111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:57.754508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.756318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.785021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 252/512] 10.222.0.30 () {56 vars in 1139 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:57.785607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.826800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48db31-3792-4fb1-a998-3ea62cebf9fe None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.874665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.875650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.876184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.876674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.877279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.877829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:57.878127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.880559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:57.902387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 261/513] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:57.903743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:57.945923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a560ff2-46c4-4982-8e13-bad8b664b48a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:57.977736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:57.991143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:57.991840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:57.992159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:57.992442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:57.993209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:57.993517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:57.995634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.011457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 253/514] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:58.012439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:58.050223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78385c7a-b75c-41ca-af7d-7de4dd136f4d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.092522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.093225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.093664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.093823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.163913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5b6e2-76de-40dc-90fd-5dad2a9042c3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.180888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.183541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.193600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.207843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.208435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.208759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.209022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.239650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.290633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.293612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.299140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 262/515] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:58.300004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:58.417432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c0f2e-1cf4-4c93-8a00-cf5438212233 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.422118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 254/516] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:58.423131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:58.451454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.470524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.471860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.472606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.473090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.500171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.568148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad95f49-bb0a-445f-8fab-a70af6331686 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.572212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.574553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.616523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.617207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.617486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.617764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.618483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.620695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.646268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 255/517] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:14:58 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:14:58.647280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:58.731530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 263/518] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:14:58.732868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:58.796840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.829776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.878282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed437-f433-4f50-98e4-7fd2e9ab38ff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.902377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514128-6f0e-4f50-9a37-8e4df1e96b9b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.947529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.949084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.949558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.950018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.957381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.958199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.958477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.958738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.971424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.973335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.974804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.975659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:58.997499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:58.998608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:59.000263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:59.000263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:59.000263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:59.000263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:59.000263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 264/519] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:59.000263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:14:59.004959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:59.006816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:14:59.009668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:59.011922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 256/520] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:14:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:59.012948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.207530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614f99-9692-438c-9aaf-c254aba7b52d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.222681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:59.227953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812558a4-ba7b-425d-98a9-27a7b449a41c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.266527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.267242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.267519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.267792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.276273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:59.283391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.287020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.296086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.296884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.297162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.297436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.310953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.312595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:59.314049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:59.315601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:59.317183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:59.317183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:59.317183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:59.317183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:14:59.317183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 257/521] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:59.317183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:14:59.339158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:14:59.341061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:14:59.342667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:14:59.342667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:14:59.342667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:14:59.342667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:14:59.342667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 265/522] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:14:59.342667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.506023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae40a-3912-4694-b6da-ceda05a71c86 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.540203 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17440ae0-4a78-4e63-8872-e99e9194271b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.550829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.551511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.551797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.552092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.552945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:59.553254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.554712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.581998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 266/523] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:59.583838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.605230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:59.615509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.616258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.616505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.616786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.617906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:59.617906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.620018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.633455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 258/524] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:59.636017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.708185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:59.755671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbc4ad-a422-4c32-986b-5e3f847fe1b5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.783419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11ebb1-0bf1-45e2-b73e-e45827991187 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.814887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.815490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.815773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.816080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.816825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:59.817202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.818850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.839990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 267/525] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:59.839990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.857348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.858345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.858778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.859218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.860406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:14:59.860893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.863124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.882326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 259/526] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:14:59.888407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.930380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:59.988855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6108b8-e270-49d9-980d-470ff8c58cdf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.045949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.047189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.047808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.048304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.054458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47497458-c126-4b64-83b1-b82400fcb08c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.072840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.113910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.115185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.115658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.116144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.129082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.131621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:00.159179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:00.161038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:00.162686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 260/527] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:00.163588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.182278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.184798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.299571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.343192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa0972-d479-48d7-9297-493f2102293b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.343901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 268/528] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:00.344928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.394097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.395338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.395759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.396234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.409597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.413033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:00.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:00.441985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:00.442993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:00.448569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 261/529] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:00.450109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.488264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb2ba3-3784-4535-8fd5-5a7cdf3daf8f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.491266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.541521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.542237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.542496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.542755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.550699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.552257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:00.569000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:00.570622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 269/530] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:00.573124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.592747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce729e5-a69d-4e30-aae9-3499d7949a12 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.612281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.637729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.638502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.638788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.639094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.639970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:00.640322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.642296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.672097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 262/531] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:00.672097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.732307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c9409-053c-441d-92b4-66ee121854aa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.778241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.779170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.779459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.779736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.793452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.795477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.797871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.811957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ea354-d019-432c-976e-42f596e69524 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:00.820611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:00.822817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:00.823969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:00.823969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:00.823969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:00.823969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:00.823969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 270/532] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:00.823969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.864078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.864735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.865033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.865302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.866200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:00.866520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.868572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.883508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 263/533] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:00.885483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.958314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.987138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30922e4c-2d02-4d4b-a96c-32cc7449a08f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.017561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe578-0dc5-489a-981f-1f7a283b911a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.037409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.038996 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.039768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.040529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.042557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.043466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.046580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.075759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.077186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.077513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.078146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.080338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=793065092530407983e5da2446d2e939) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.080883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.083390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.092143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 271/534] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.092143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.096103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 264/535] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:00 2022] GET /identity/v3/role_assignments?user.id=793065092530407983e5da2446d2e939&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.097259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.133135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:01.277471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f537d87-c3a3-4e93-9a57-ac7570b19606 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.303087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ef23-5c1b-48b5-b9f3-0d85788a0fcd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.351594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.352813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.354107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.354798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.355954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.356592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.358432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.374453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.375267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.375624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.376220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.377408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=558f4050abe8467aa527d73a49f101b1) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.377802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.379527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.388345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 265/537] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/role_assignments?user.id=558f4050abe8467aa527d73a49f101b1&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.388732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 272/537] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.389275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.403990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.409534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:01.530098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:01.562398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88b186-1107-4cb0-a964-bfb4d468a777 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.562790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df10f2-97d8-4ec0-85b9-270cde95a744 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.607965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.609083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.609461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.609663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.609800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.610916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.611425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.612109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.613243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=80ea2163e08f4443a3d6af03a59351f2) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.613816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.615764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42f5dc-219d-4a40-82ca-3dedbce917b9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.618754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.621335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.624924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 266/538] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/role_assignments?user.id=80ea2163e08f4443a3d6af03a59351f2&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.626901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba32d9-4fb9-41ea-baac-2263996c9aa9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.627511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba32d9-4fb9-41ea-baac-2263996c9aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.628071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba32d9-4fb9-41ea-baac-2263996c9aa9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.630187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 267/539] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:01.640102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.640102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.640102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:01.642916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:01.643834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:01.644823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 273/540] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:01.653896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.673252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:01.674544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:01.796973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:01.809940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 268/541] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:01.859996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.874128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf9236-8cd8-428d-b06e-2efe1357bc4a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.938412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.939077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.939369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.939706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.941175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.941734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.944413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.970819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 274/542] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.977237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.044454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:02.110761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa55f4-d5a5-4720-a1da-e67da8696825 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.212895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490c53c-ad63-4b82-9a7e-f018a26ddd7e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.237457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.238299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.238584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.238861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.248089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:02.265376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.266364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.266688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.267051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.268329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=49b361e969094db296e3fc02eda18dc3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:02.268776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.271021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.281670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 275/543] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/role_assignments?user.id=49b361e969094db296e3fc02eda18dc3&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:02.281914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:02.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:02.306783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:02.309139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 269/544] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:02.316660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.513876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a2f9-0fb8-467a-a04a-af683ce3480b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.576203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.577847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.578455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.579138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.580692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:02.581418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.583791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.602543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 276/545] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:02.609009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.782357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb80280-fa08-4650-b047-b247b90df37b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.833025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:02.840328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.841232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.841525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.841821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.855890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.858181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:02.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:02.883089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:02.885050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:02.885050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:02.885050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:02.885050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:02.885050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 270/546] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:02.891144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.015424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:03.097720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d09cf-198f-4540-8b45-891ff629a16c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.176616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.177325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.177607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.177881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.178642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:03.178953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.180608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.200087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 277/547] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:03.206692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.456583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83127-410c-4659-a345-e5241d75d441 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.559168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.560014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.560316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.560602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.577539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.579053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:03.605072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:03.607479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:03.609503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 271/548] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:03.612245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.724321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1191006-e9a4-4aa3-aa0d-ea2af16a84df None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.736524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:03.762598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.763406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.763690 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.763967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.764872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:03.765180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.766629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadf57e-c318-4904-86f9-375bd70cfc6c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.786768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 278/549] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Jun 16 18:15:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:03.792936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.866537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:03.946284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639691d-c6f5-435f-b44d-7db59112e0df None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.977786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b7f1-82f8-4eed-a615-a7ac03204512 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.978639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b7f1-82f8-4eed-a615-a7ac03204512 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.978851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b7f1-82f8-4eed-a615-a7ac03204512 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.980693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 279/550] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:03.986761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.987077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.987412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.010143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:04.014004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:04.015122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.015624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.016158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.018430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:04.210720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 280/551] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:15:04.233938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:04.236080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:04.314151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:04.332468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:04.420535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 272/552] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:04.425668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:04.499684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:04.505905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee9fc9-ab41-4057-bea5-6065bd6ef824 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:04.549265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fcb80-2800-45bc-ab4b-ec4c5c4c7fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:04.550378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fcb80-2800-45bc-ab4b-ec4c5c4c7fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.550872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fcb80-2800-45bc-ab4b-ec4c5c4c7fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.551409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fcb80-2800-45bc-ab4b-ec4c5c4c7fd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.564990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fcb80-2800-45bc-ab4b-ec4c5c4c7fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:04.571839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0960fcd-f149-42b1-a3e6-d225e847564a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-324fcb80-2800-45bc-ab4b-ec4c5c4c7fd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:04.574195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:04.575103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:04.575932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:04.578270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 281/553] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:04.579877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88955a-dfa5-46ab-b2cf-b0b59ad24f32 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.580400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88955a-dfa5-46ab-b2cf-b0b59ad24f32 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.580850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88955a-dfa5-46ab-b2cf-b0b59ad24f32 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.587209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 282/554] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:04.588648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88955a-dfa5-46ab-b2cf-b0b59ad24f32 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:04.619793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:04.620669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.620985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.621274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.636660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:04.638265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:04.660530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-388875e1-d306-4da8-8cc0-a02ec16342f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:04.669807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:04.678042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 273/555] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.679735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.708096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:04.710329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:04.747206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88955a-dfa5-46ab-b2cf-b0b59ad24f32 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:04.796753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:04.797348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.797634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.797911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.798673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:04.798984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:04.800419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cec4d-182e-4d3a-b6fe-ac3bc6280be3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:04.805720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 274/556] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:04.808110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:04.816669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 283/557] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:04.818545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9eb78c-832c-46a0-bd94-386bc8468b24 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:04.819123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9eb78c-832c-46a0-bd94-386bc8468b24 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:04.819862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9eb78c-832c-46a0-bd94-386bc8468b24 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:04.821597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 284/558] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:04.828879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9eb78c-832c-46a0-bd94-386bc8468b24 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:04.858998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:04.964684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9eb78c-832c-46a0-bd94-386bc8468b24 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:04.968939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af7f93-0346-4bb6-833d-e196a70adc69 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.007533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.016730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.017772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.018146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.018503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.019519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.020290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.020578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.020921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.021901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:05.022359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.024000 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7881862-5b2c-4f29-ad2c-2e079ce42dc5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.037323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.038969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.043689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 275/559] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:05.045511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.045816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.046163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:05.062514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:05.063673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:05.065811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:05.065811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:05.065811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:05.065811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:05.065811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 285/560] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:05.065811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.077999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:05.079613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:05.174638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.187871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 276/561] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:05.188824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.223576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cd3be-2df5-475e-a98a-418cba971373 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.265733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.266604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.266889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.267196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.288689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:05.298832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:05.299958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 286/562] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:05.300840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.309240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1fbb1-91a5-485b-a7a1-43facc25e282 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.329224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.350985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.351907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.352276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.352575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.360562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.362046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:05.377176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:05.379002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:05.380606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:05.383406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 277/563] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:05.384917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.430926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec59ae9-1f06-419f-838d-2ffcf0f865cb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.468102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.475249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.475985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.476347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.476620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.477381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:05.477695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.479911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaae903-52d6-4aeb-bb9c-bbb7c951f197 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.495575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 287/564] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:05.497014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148e5a-8c9c-480e-8e99-902730d60163 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.497300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148e5a-8c9c-480e-8e99-902730d60163 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.497579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148e5a-8c9c-480e-8e99-902730d60163 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.498970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 288/565] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:05.499740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148e5a-8c9c-480e-8e99-902730d60163 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.531405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb2e70-2865-4c19-8395-8d691c20c0a3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.578020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.579032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.579557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.579886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.592902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:05.601560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:05.603756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 278/566] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:05.604688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.644348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.654200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148e5a-8c9c-480e-8e99-902730d60163 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.699421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.700133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.700419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.700697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.701456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:05.701767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.703148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.711725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 289/567] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:05.713125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.743707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d35e2c-fa2c-483b-a5a8-b2d1c82a96fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.776659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.786085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.786779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.787074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.787357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.788252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:05.788611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.790275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486504-f8fb-47bb-ae80-c746c7a450a7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.805920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 279/568] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:05.808256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.808579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.808908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.828174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:05.829814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:05.840676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523885ff-2f51-48d8-a33f-43fbc2fdcb77 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.879385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.880498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.880781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.881135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.891234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.892732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.911439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 280/569] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:05.912314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:05.918693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:05.919712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:05.920695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:05.929803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 290/570] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:05.931999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.944265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.036116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d2c7d-cfb9-4a8c-95af-ad30756024a4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.068645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e3916-cd12-4304-bf22-da0645b4fa4b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.088325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.097448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.098216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.098527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.098841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.099694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:06.100039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.101815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.113344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 281/571] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:06.114209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.115079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68971600-f102-448b-9601-d2fa0430085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.115902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68971600-f102-448b-9601-d2fa0430085e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.116433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68971600-f102-448b-9601-d2fa0430085e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.116620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68971600-f102-448b-9601-d2fa0430085e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.130362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68971600-f102-448b-9601-d2fa0430085e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.132350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68971600-f102-448b-9601-d2fa0430085e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-68971600-f102-448b-9601-d2fa0430085e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:06.152096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:06.153614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:06.154554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:06.154554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:06.154554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:06.154554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:06.154554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 291/572] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:06.154840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68971600-f102-448b-9601-d2fa0430085e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.227983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.242338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f311-345e-49ee-a730-2d863a5a8073 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.291440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.292472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.292797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.293209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.294226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=04b5081e9bf640679c72beb1b18f3e78) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:06.294592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.296445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.301110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68971600-f102-448b-9601-d2fa0430085e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.302834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 282/573] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/role_assignments?user.id=04b5081e9bf640679c72beb1b18f3e78&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:06.303668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.360399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.361302 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.361715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.362160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.363420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:06.363812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.366115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.407976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 292/574] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:06.407976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.451939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.464489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70439f-3d21-496b-b22c-41884ba10bb8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.536623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.538922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.540127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.541909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.562952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:06.567646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f193b-7e25-4dfb-99a3-6a4663dcddf1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:06.574644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:06.575722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:06.576750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:06.577777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 283/575] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:06.578754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.600341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.627391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.628551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.628889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.629376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.650062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.653201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:06.682016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:06.683473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:06.684921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:06.684921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:06.684921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:06.684921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:06.684921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 293/576] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:06.684921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.719957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed26b-a19c-48eb-b9b5-9118f3d5b62c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.767859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.768709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.769102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.769432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.770313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:06.770729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.772949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.796096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 284/577] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:06.796096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.807570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.848491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87f84d-a8ae-4635-a57e-1836e3bee5df None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.901767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.902528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.902783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.903060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.907018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:06.916369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.918066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:06.938212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:06.939860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:06.940915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:06.940915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:06.940915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:06.940915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:06.940915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 294/578] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:06.940915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.965092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dee92-ad4b-4584-92a3-7a1620f8011f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.023634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.025426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.026097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.026458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.027820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:07.029865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.032949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.050213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 285/579] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:07.051543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.082051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9258b-3266-439b-aef8-d9fccea4b6e9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.123609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.139058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.140019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.140449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.140884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.142148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:07.142656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.144626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.170212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 295/580] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:07.171862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.207572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c432c3-7f10-4a45-94d8-80716ef65dcd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.258807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.260296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.260647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.260924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.263217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.272319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.273804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:07.291455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:07.295162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 286/581] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:07.297802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.333569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46c87a-1941-40f1-841a-c0adb524385b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.390125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.390864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.391182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.391496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.392445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:07.393017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.395191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.415218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 296/582] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:07.416430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.467732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f585d9b-a670-47f6-9d5b-e6e6f56e0785 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.499551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.536742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.537670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.537849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.538127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.552003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.553654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:07.572760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:07.573751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:07.574656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:07.574656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:07.574656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:07.574656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:07.574656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 287/583] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:07.575293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.576590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5195e-a033-4738-98eb-bbefe968ea20 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.621531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.622362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.622700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.622974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.629728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.632864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.635220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:07.653181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:07.654201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:07.655133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 297/584] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:07.655960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.716737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05d3f6-33c7-481c-8b9b-e4a874cb8051 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.761911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.762682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.763022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.763455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.764358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:07.764677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.766206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.804076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 288/585] 10.222.0.30 () {56 vars in 1139 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:07.804280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812450-36a0-41e0-90e9-226638a2cecb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.804379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.815876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.852556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.853958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.854281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.854563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.862401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.864117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:07.879886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:07.881195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 298/586] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:07.883037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.945828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720bf3-1694-4250-98e2-4368278e4cbd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.949926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.019865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.020780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b7f9f-71ab-4487-8660-925858004346 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.020991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.021470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.021973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.023750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:08.024515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.029024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.051868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 289/587] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:08.053425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.066086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.066806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.067091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.067400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.068312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:08.068638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.070152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.098194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 299/588] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Jun 16 18:15:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:08.098194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.129323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.221626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3d83d-ef74-4a41-b734-53b0b7bad004 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.243743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e672f-4af0-49e4-8f7f-f083c3bf2741 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.269038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.269731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.270121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.270304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.271075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:08.271384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.272859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.287600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.288450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.289049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 300/589] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:08.290190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.294209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.294573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.308338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.309836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:08.328770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:08.329820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:08.330711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:08.330711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:08.330711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:08.330711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:08.330711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 290/590] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:08.330996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.358887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.410725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.432483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd81987-1621-4397-95b3-54c9b49ff317 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.473669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c804f-8a59-4b94-9955-05dbb9cc5814 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.488098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.489643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.490264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.490870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.506345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.510370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.531465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.532644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.533091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:08.533951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:08.535751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:15:08.539585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:08.543879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.544150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 301/591] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:08.544150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.563260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.565250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:08.586622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:08.589174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:08.591321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:08.591321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:08.591321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:08.591321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:08.591321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 291/592] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:08.591321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.676552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f6d2-64c6-481a-9e0e-6b08ae7d1641 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.684342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.721337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.723256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.723619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.724094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/793065092530407983e5da2446d2e939/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.735479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108a592-6f2c-4544-98f0-846850055ef8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.745588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=793065092530407983e5da2446d2e939, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.747244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.749960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.767858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 302/593] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:08 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/793065092530407983e5da2446d2e939/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:08.768823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.815662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.816407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.816693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.816970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.817726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:08.818036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.819699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d6051-abcd-4024-abf5-983dd1c5ff73 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.834863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 292/594] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:08.836081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8ff01-aa15-46f6-bf95-14fdb3ff57ac None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.836383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8ff01-aa15-46f6-bf95-14fdb3ff57ac None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.836658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8ff01-aa15-46f6-bf95-14fdb3ff57ac None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.844457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 293/595] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:08.845470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8ff01-aa15-46f6-bf95-14fdb3ff57ac None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.939140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c2c9d-64b4-4c51-bf4b-9801199ed4bc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.986412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8ff01-aa15-46f6-bf95-14fdb3ff57ac None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.987812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.988834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.989287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.989626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.990701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:08.991101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.993014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.027370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.031513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 303/596] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:09.033877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.051803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.052716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.053139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.053724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.054594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:09.055050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.057431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b199c9d-d36f-4f76-b7d0-4e98d0d556f3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.083354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 294/597] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:09.083354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.083354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.083354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.103850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.111670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:09.112954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:09.178201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cb07-1f6c-45a1-b2ba-5373a62a7427 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.196092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 295/598] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:09.196330 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.235880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.237194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.237660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.238122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.247625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.249128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:09.263666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:09.264838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 304/599] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:09.265754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.355737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cda65-fb80-4137-80d1-7b3a70b9086c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.381939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.409053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.409930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.410221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.410527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.415922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a8701-279d-4990-9a0b-a41781bc1a4d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.423427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.425077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.433706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:09.445682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:09.447600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:09.448836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:09.448836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:09.448836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:09.448836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:09.448836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 296/600] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:09.448836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.472659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.473476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.473762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.474105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.482487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.485076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:09.504915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:09.506681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:09.508478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:09.515884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 305/601] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:09.515884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.602256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78ba36-0da5-4725-a070-bf5b9a86daa2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.642028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.642759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.643046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.643319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.648425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51b0aa-b502-4733-b4ca-d3e3f7798f2d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.654939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:09.663103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:09.664546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:09.666501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:09.671437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 297/602] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:09.672308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.696606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.698131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.698594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.699118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.700205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:09.700835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.702872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.708766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.718536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 306/603] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:09.719131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.764562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.817808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9c08b-b1ad-4f0f-b86a-7762a59f1228 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.862542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6a29-16c5-4d55-bf0f-12dc74d18414 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.866683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.867499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.867820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.868101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.869534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:09.869715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.871179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.889951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 298/604] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:09.891062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.923039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.923646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.923957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.924258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.925092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:09.925432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.927048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.943506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 307/605] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:09.944408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.014499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.029845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480382d-f04c-4110-9e01-64320ce5360e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.074273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.074967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.075251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.075529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.076415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:10.076883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.078477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.088734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 299/606] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:10.094015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.096101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.114544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaa574-3419-424d-a8cc-92cca154eb94 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.166016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.167475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.167966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.168330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/558f4050abe8467aa527d73a49f101b1/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.191316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=558f4050abe8467aa527d73a49f101b1, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.193227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.220084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 308/607] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:09 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/558f4050abe8467aa527d73a49f101b1/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:10.220084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.256405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a005-ba95-4cfe-b9bf-ca954bcd91cb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.309979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.310959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.311312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.311656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.320962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.324202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:10.348337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:10.349287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:10.350185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 300/608] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:10.350754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.375381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.382172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc70-e56f-45bf-b626-4834667f3cd1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.450975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.451825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.452137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.452418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/80ea2163e08f4443a3d6af03a59351f2/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.466803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.469998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=80ea2163e08f4443a3d6af03a59351f2, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.473048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.494219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 309/609] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:10 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/80ea2163e08f4443a3d6af03a59351f2/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:10.495424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.519253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e71a1-a010-4c58-a6bc-bf741f666fb7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.561099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.561934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.562216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.562494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.575689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.577231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:10.598538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:10.599630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:10.600850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:10.600850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:10.600850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:10.600850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:10.600850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 301/610] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:10.610532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.756269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6a090-1c60-4c26-a40a-bdbeb3e7d729 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.770612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.816783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.818014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.818516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.819388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.820651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:10.821376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.824174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.833592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab2f9-2925-4b18-a307-c3ce46a60cb5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.847036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 310/611] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:10.854061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.904595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.907563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.907874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.908200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.909746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:10.910060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.911468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.942070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 302/612] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:10.948547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.953900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:11.044275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75550cff-bfd4-4018-907a-015bb74e721c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:11.100085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ad34e-ba07-453a-8262-60b9508dfc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:11.101056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad34e-ba07-453a-8262-60b9508dfc82 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.101686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad34e-ba07-453a-8262-60b9508dfc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.102299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad34e-ba07-453a-8262-60b9508dfc82 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/49b361e969094db296e3fc02eda18dc3/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.126660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ad34e-ba07-453a-8262-60b9508dfc82 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=49b361e969094db296e3fc02eda18dc3, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:11.129494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ad34e-ba07-453a-8262-60b9508dfc82 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:11.152446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 311/613] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:10 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/49b361e969094db296e3fc02eda18dc3/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:11.154513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877f3cb-4880-4983-bf2c-d30f6ee66f46 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.154802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877f3cb-4880-4983-bf2c-d30f6ee66f46 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.155107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877f3cb-4880-4983-bf2c-d30f6ee66f46 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.162731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 312/614] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:11.165323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.166428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.166788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.189049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed78dea-3474-4829-b942-8e023da3d996 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:11.191510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:11.193902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:11.247832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:11.265045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:11.265875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.266154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.266426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.277612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:11.279465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:11.305796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:11.309477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:11.312705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 303/615] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:11.319659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:11.338077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 313/616] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:15:11.398912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:11.459437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:11.496164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6c8f-1f48-4711-ad14-fcb2e4552fd0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:11.577649 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:11.578281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.578633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.578992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.580025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:11.580777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:11.582468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:11.623278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 304/617] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Jun 16 18:15:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:11.629140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:11.635787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bca107-a44c-405b-83f7-0dbac38d6720 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:11.706337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:11.719037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:11.719863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.720171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.720453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.752664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:11.761905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:11.762917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:11.763818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 314/618] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:11.772073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:11.846143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f3ba6-2f59-4e94-a70d-72c9190abeda None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:11.927710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:11.929045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.929422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.929708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.957600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:11.959224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:11.996777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:11.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:11.998899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:11.998899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:11.998899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:11.998899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:12.008853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 305/619] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:12.010441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:12.012393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:12.020778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b7381-83a8-49e7-9ad2-c3da662490db None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:12.071249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:12.072527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:12.072968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:12.073277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:12.074059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:12.074492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:12.075981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:12.087102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 315/620] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:12.092835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:12.224892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba64e7-d62a-4683-93bb-b6d055dabd74 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:12.257973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:12.269606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:12.270426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:12.270715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:12.270951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb80092-1d5c-438f-bdcd-202ddf52ab96 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:12.271061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:12.285006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:12.286559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:12.307960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:12.308960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:12.309867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:12.309867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:12.309867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:12.309867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:12.309867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 316/621] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:12.315153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:12.326437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:12.381755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:12.382444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:12.382729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:12.383002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:12.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:12.384061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:12.386292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:12.419710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 306/622] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:12.429775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:12.451440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45033-ee00-4590-afcf-2c8f6f042ae8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:12.497445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:12.498062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:12.498347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:12.498627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:12.499377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:12.499688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:12.501140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:12.518710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 317/623] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:12.524384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:12.613500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:12.660003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0158b-eedc-4d5a-9b9f-5f3a12410c9d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:12.707129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:12.708589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:12.709130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:12.709892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:12.724380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:12.726731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:12.760618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:12.764837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:12.769753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 318/624] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:12.779468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:12.780199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631924b4-bf8b-469d-9d86-28be55ee6759 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:12.885882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:12.887209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:12.887564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:12.887866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:12.911753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:12.913822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:12.955519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:12.956558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:12.957468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:12.959537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 307/625] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 530 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:12.965605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.000310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:13.020246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79a986-f8dd-47d7-938b-267550903cdb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.113127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:13.134695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.136277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.136741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.137252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.139229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:13.139806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:13.144189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51163704-0b6f-49e3-8d1f-fa762f45c55e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:13.183430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 319/626] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:13.183430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c389-9493-4ba3-aaf5-61b6eaa10d03 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.183430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c389-9493-4ba3-aaf5-61b6eaa10d03 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.183430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c389-9493-4ba3-aaf5-61b6eaa10d03 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.184751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 320/627] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:13.188080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4c389-9493-4ba3-aaf5-61b6eaa10d03 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.196186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d6a56-ab0d-4227-8a87-c0b8c6ca97de None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.250876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.251891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.252354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.252777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.254057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:13.254557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:13.257365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b35fe2-e76c-436b-86a0-bc042237856e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:13.275432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 308/628] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:13.277362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.277656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.277958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.302548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:13.303873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:13.342594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4c389-9493-4ba3-aaf5-61b6eaa10d03 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.398567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.399353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.399623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.400648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.409727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 309/629] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:13.411184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.416138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:13.490640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:13.517189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:13.517937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:13.558135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9a98-7ddc-49be-937a-ed84c1f020f7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.626829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c8da-43fd-4af1-99b0-943ce062924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.628792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c8da-43fd-4af1-99b0-943ce062924b None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.629524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c8da-43fd-4af1-99b0-943ce062924b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.630229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c8da-43fd-4af1-99b0-943ce062924b None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/04b5081e9bf640679c72beb1b18f3e78/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.653025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c8da-43fd-4af1-99b0-943ce062924b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=04b5081e9bf640679c72beb1b18f3e78, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:13.654588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c8da-43fd-4af1-99b0-943ce062924b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:13.668597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 321/630] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:13.680087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.680323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 310/631] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:13 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/04b5081e9bf640679c72beb1b18f3e78/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:13.680323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33da37-4249-4923-ad49-20385f95142e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.680323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33da37-4249-4923-ad49-20385f95142e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.680323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33da37-4249-4923-ad49-20385f95142e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.680323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 311/632] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:13.680323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33da37-4249-4923-ad49-20385f95142e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.784448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:13.854308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:13.862023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33da37-4249-4923-ad49-20385f95142e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.880433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624b5a-351b-4f93-871f-8582a0aa5781 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.908229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.909302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.909751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.910030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.924421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:13.925953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:13.926908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.928009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.928610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.929125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.944809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-239dd2bd-c443-4168-93a5-ece1f0e9b15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:13.948798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:13.949697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:13.950598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:13.950598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:13.950598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:13.950598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:13.953076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 312/633] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:13.953217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:13.954080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:13.954945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:13.957479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.957980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.958439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.959131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 322/634] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:13.960019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.987520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:13.989308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:14.080742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 313/635] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:14.081600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.089029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a218851-533c-4159-a6b6-77c2dad153f5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.140198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.141188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.141661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.142143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.143082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:14.143580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:14.145199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:14.151549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:14.163207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 323/636] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:14.164491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.169333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:14.248823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e7c8-bf66-4147-8bb6-22952281a878 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.291587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.292341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.292630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.292905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.293716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:14.294073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:14.295656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:14.308460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 314/637] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:14.310684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.320737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457f639-7773-446e-8fc0-5d4a31580ea2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.364627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd8027-3d90-4051-ab91-04c8c031a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.365458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8027-3d90-4051-ab91-04c8c031a31b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.365626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8027-3d90-4051-ab91-04c8c031a31b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.366076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8027-3d90-4051-ab91-04c8c031a31b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.379420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd8027-3d90-4051-ab91-04c8c031a31b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4bfd8027-3d90-4051-ab91-04c8c031a31b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:14.388897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:14.396878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 324/638] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:14.401575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be166cd-744c-48ba-a4ae-fc6a6395971a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.406207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be166cd-744c-48ba-a4ae-fc6a6395971a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.406207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be166cd-744c-48ba-a4ae-fc6a6395971a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.406207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 325/639] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:14.406207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be166cd-744c-48ba-a4ae-fc6a6395971a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.462851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:14.467816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf313-89b9-4ce8-a96d-dbf2f5a5c9ce None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.491257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:14.518077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.519359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.519816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.520184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.531960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:14.533727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:14.552021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:14.553294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:14.554573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 315/640] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:14.556609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.560277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be166cd-744c-48ba-a4ae-fc6a6395971a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.613026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.614372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.614948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.615301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.630926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:14.633252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-43462b54-83d5-4e8e-89b9-0d44cb65351f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:14.655709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:14.656791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:14.657713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:14.657713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:14.657713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:14.657713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:14.657713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 326/641] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:14.658436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.658777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.659055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.681681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:14.683311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:14.730521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551bbbd-6a98-4bf0-95e3-7779f97adedb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.783794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.784956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.785611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.785951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.786913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:14.787444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:14.789603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:14.800133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 327/642] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:14.800133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.818387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 316/643] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:14.819674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:14.847370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:14.856142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:14.925705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e40f06-d968-434c-b531-3dd6c0fd2315 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.956071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d118d9-14f5-4ab8-a3f7-e3589c455e86 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:14.976030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:14.976730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:14.977101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:14.977344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:14.978372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:14.978754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:14.980410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:14.995600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 328/644] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:14.996534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.026954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.029424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.029833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.030173 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.031426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:15.031831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.033787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.053855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 317/645] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:15.059452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.154725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e260e-41e3-4d07-9565-dc04d764ee0c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.167648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:15.179129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:15.202868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7a36d-f778-4703-8415-df35b68ec70c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.210268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.211293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.211581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.211880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.225993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.227525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:15.250747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:15.251839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:15.253571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:15.253571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:15.253571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:15.253571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:15.253571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 329/646] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:15.253571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.262844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.263803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.264129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.264469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.281370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:15.291194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:15.293359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 318/647] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:15.295062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.392316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99718a5-28eb-4960-8c49-b514ed3098ce None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.426035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcbf2a-973a-45e7-b221-83b4d97f7c4e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.431768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.432567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.432850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.433124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.433982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=793065092530407983e5da2446d2e939) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:15.434296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.435781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.442897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 330/648] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/role_assignments?user.id=793065092530407983e5da2446d2e939&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:15.443818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.481920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:15.497367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.499815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.500936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.501410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.513317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.515122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.518778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:15.537416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:15.539060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:15:15.540141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:15.543711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 319/649] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:15.545270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.604429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9f9f-839b-4e53-9f0f-80c5b0490413 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.667270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.668077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.668492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.668763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.669864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:15.669864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.671862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.687206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630170e-5a35-44ff-af00-2129ab1ffe12 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.689319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 331/650] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:15.690704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.743880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.745253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.745752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.746228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.747070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:15.747392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.748952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.763511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 320/651] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:15.764599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.813163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:15.840259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d09747-c7d9-4882-a52b-8d9ebcef1153 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.882697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:15.885700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.887012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.887751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.888335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:15.890304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:15.891028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:15.894209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:15.921356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 332/652] 10.222.0.30 () {56 vars in 1139 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:15.926849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:15.945142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334eae-fd4f-46e7-b789-f604231d48f8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:15.995783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:15.997055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:15.997575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:15.998102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.008690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.011318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:16.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:16.029610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:15:16.030811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:16.038290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 321/653] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:16.039993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:16.064886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7422f67-ec66-4379-9826-8549ef7d623f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:16.108300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:16.109342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.109716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.110051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.121667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:16.124558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.126608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:16.147810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:16.148927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:16.150106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:16.150106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:16.150106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:16.150106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:16.151887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 333/654] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:16.152877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:16.184789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0e63-3d4c-462b-891a-4dd7306bb2fc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:16.219501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:16.236692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:16.237518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.237814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.238169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.287427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbeb78e-19d8-4493-853d-15ed3558c949 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:16.323562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.325257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.336509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:16.338133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.338421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.338704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.339605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:16.339942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.341573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc17c-b0f6-4839-8119-b202e936f37f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.366859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 334/655] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:16.368488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df69644-085a-4d98-b2b2-26a383f435a5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.368974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df69644-085a-4d98-b2b2-26a383f435a5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.369370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df69644-085a-4d98-b2b2-26a383f435a5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.373990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 335/656] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:16.375400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df69644-085a-4d98-b2b2-26a383f435a5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:16.460130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:16.486586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 322/657] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:16.486586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:16.525293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:16.540145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df69644-085a-4d98-b2b2-26a383f435a5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:16.585772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:16.586464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.586741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.587021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.589261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:16.589577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.591046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8dde7-1516-4d65-849f-a655863f5d35 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.605380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 336/658] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:16.607098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.607445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.607791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.627135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:16.628475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:16.633136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fe289-3c6b-4f2e-bb15-d4cc0d694b88 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:16.678452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:16.679466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.679752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.680958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.709646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 337/659] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:16.711110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:16.772021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.773317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:16.774220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.811340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:16.869398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969136d-1344-4126-8d1b-0469f199d63b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:16.907403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 323/660] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:16.909156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:16.934321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:16.935229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:16.935572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:16.935905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:16.947766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:16.948927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:16.969531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:16.971265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:16.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:16.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:16.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:16.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:16.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 338/661] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:16.973071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.052850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307534a-4d63-41d7-b47c-8c7352b1849d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.092943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:17.119526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.121478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.122013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.122431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.131833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:17.135222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.137446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.148807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1047f-1bbe-4fa9-b432-52a44babec65 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:17.156011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:17.157941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:15:17.159264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:17.162534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 324/662] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:17.163636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.198179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.199194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.199571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.199976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.217139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:17.230267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:17.231293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:17.232233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:17.232872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 339/663] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:17.233612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.303563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4caa-2ff6-447f-ad19-c0bdc4473c70 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.356758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.357747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.358131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.358428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.380272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf0110-d4f8-4dbc-afb8-fa6e67363dcb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.380518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.382075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:17.413377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:17.415089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:17.417010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:17.417010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:17.417010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:17.417010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:17.417010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 325/664] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:17.417010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.443030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.443859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.444262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.444616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.445373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:17.445685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.448074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.461648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:17.466704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 340/665] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:17.467565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.484244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:17.562832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dd88d-7a24-4ed1-8005-aa7ec4613902 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.609675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.611091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.611655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.612177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.613593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:17.614154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.616235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.616708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e2fac-f06a-41a0-bb42-e8593b233a6a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.645028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 326/666] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:17.645028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.668024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.669208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.669631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.670089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.670775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:17.671087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.672807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.689336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 341/667] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:17.690695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.774404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:17.785663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cbaa8-22f2-4506-9c8c-6488128ae084 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.797268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:17.829250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.829942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.830233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.830505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.831296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:17.831605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.831972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47f730-bca3-4c2d-b98b-68efc23a59ba None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:17.833020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.853442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 327/668] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:17.854279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.885247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:17.886236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:17.886712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:17.887114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:17.898250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:17.901248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:17.928200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:17.929429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:17.930385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:17.930882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 342/669] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:17.932826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:17.989917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ba0a3-a815-453a-aecd-7f62c9000f88 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.039673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.040411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.041287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.041602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.065010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3f804-ef85-4691-8d47-8275af899d97 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.070390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.106019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.106745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.107014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.107297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.109583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.119743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.121440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.129289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.130828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:18.142442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:18.144156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:18.145879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:18.145879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:18.145879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:18.145879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:18.145879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 343/670] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:18.145879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 328/671] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:18.252806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.282748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdddab8-0cee-4910-8fd9-20d844dea0d8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.327183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.327999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.328450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.328604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.329532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.338039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.340311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-11043d2f-fc8f-4c0e-8e94-ccda6dbe88e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:18.355039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:18.356301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:18.357263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 344/672] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:18.357754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ec329-91b8-4844-8a7c-554e08bde33c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.358004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ec329-91b8-4844-8a7c-554e08bde33c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.358270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ec329-91b8-4844-8a7c-554e08bde33c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.359523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 345/673] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:18.360259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ec329-91b8-4844-8a7c-554e08bde33c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.402027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b0f9c-6e2c-4b11-84f7-d51775c1e00c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.421404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.459943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.460654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.460938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.461224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.462033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:18.462924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.464918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.487082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 329/674] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:18.487082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.499712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ec329-91b8-4844-8a7c-554e08bde33c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.546525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.547670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.548180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.548666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.549871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:18.550222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.552703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.569553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 346/675] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:18.570676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.603916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9648c5a-ecf8-4fc8-956c-0f2a29994a55 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.624492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.653192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.654010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.654290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.654567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.667846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.669583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2044e525-ccea-46be-a23d-42db79ba94af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:18.692278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:18.694094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:18.696722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:18.696722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:18.696722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:18.696722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:18.696722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 330/676] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:18.696722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.697910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.698306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.707904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813415f3-abe1-4aa5-9f08-6709cce309c1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.717779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.718996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:18.720423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:18.752497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.753396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.753803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.754205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.755397 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:18.755828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.758024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.773387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 347/677] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:18.774244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.801081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 331/678] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:18.801938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.923583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:18.927287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf5f1-532d-4732-9129-0810939c24d5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.932515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d41b5-7603-4e5f-ba53-886850d13d36 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:18.965152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.965847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.966144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.966435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:18.967346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=558f4050abe8467aa527d73a49f101b1) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:18.967660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:18.969221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:18.987571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 348/679] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/role_assignments?user.id=558f4050abe8467aa527d73a49f101b1&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:18.987571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:18.992638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:18.994230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:18.994618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:18.994998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.008452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.011210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.025186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:19.033183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:19.034580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 332/680] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:19.035665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.147671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a25ec-c8e4-4840-be19-4a07f21fa8fc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.196862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba70bf3-385f-4c8d-9a50-7e98e228b214 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.205926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.207061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.207638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.208214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.209749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:19.210307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.213077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.233930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 349/681] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:19.235930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.256140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.257110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.257395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.257933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.258869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=80ea2163e08f4443a3d6af03a59351f2) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:19.259748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.261517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.276127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 333/682] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/role_assignments?user.id=80ea2163e08f4443a3d6af03a59351f2&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:19.276127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.307925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:19.412522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:19.430271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02e00-82e1-4c50-87a6-3b9c7038c99b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.450457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e627ed-c854-4df5-b3b2-9f253ac56c2b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.493978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.495801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.496458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.496832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.508922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.509884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.510254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.510539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.512676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:19.512676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.513210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.523392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:19.533939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:19.534941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:19.535792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 350/683] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:19.536557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.547942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 334/684] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:19.549541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.676742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:19.704483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ec9c4-d1b1-4840-a47c-ca97e2a37e29 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.714509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca714b9-732a-439d-9fd3-a9cebda7a64c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.745520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.746397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.746786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.747168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.748200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:19.748703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.750232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.761941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 335/685] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:19.763787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.765649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.766467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.766745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.767075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.775589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.777177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:19.794020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:19.794992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:19.795883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:19.800341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 351/686] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:19.801627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:19.814935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:19.912085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497cec-3ebb-47e6-ac6b-569f72388311 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.918600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e2f9b-13ff-484e-90b5-218f75ab086e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:19.960272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.961026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.961551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.961814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.975373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:19.976906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:19.991895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:19.993031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:19.994035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:19.994541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:19.997744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:19.998702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:19.999691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:19.999691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:19.999691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:19.999691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:19.999691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 352/687] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:20.000417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.021143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:20.096886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.098796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.109704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:20.166491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde412ba-c5c8-4bd4-8160-6846ae4aa636 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:20.214050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:20.214662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:20.215144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:20.215435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:20.216656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:20.217182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.219274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.238511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 353/688] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:20.239403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.256152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 336/689] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:20.257569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.318438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:20.392796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b74f-e89e-40b4-bac9-2f40eed904a2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:20.398991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063538-c41e-4d24-acf3-51f4e9a4a06e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:20.429946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:20.437281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:20.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:20.438375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:20.438652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:20.439536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:20.439851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.441529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.446237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:20.447827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:20.448197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:20.448689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:20.450092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=49b361e969094db296e3fc02eda18dc3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:20.450600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.452610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.457703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 354/690] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:20.458786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.460429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 337/691] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/role_assignments?user.id=49b361e969094db296e3fc02eda18dc3&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:20.463003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.623925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7cd9ed-931d-40c1-bde4-eaf8afdeb7f7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:20.626558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:20.679761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:20.681113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:20.681403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:20.681681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:20.701005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.702482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:20.726359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:20.727466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:20.729335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 338/692] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:20.732382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd1661-1515-495a-9aad-f2d0f883f268 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:20.741108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.795659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:20.796879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:20.797375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:20.797915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:20.811064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.813284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:20.836672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:20.838081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:20.839467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:20.839467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:20.839467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:20.839467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:20.842435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 355/693] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:20.850255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.890707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c4f69-078b-4f2c-804a-68926bcc680d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:20.907023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:20.936142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:20.937109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:20.937343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:20.937639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:20.938855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:20.939196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:20.940810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:20.963053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 339/694] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:20.969296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:20.998475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6c991-08ff-4da8-975e-d4a84ebc7d89 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:21.011519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:21.049795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:21.051000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.051604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.052203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.053412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:21.053844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:21.055782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:21.088033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 356/695] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:21.103592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:21.132607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97393713-f1ee-4dc4-ac78-fc00c506c8ef None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:21.178489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:21.179553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.180100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.180668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.298263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:21.300299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:21.328686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:21.405109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:21.422412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 340/696] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:21.427582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a0de8-d484-4d1d-99f3-0424989926bb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:21.428965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:21.492659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:21.494841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.495310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.495763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.506469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:21.509601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:21.531104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:21.532222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:21.533164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:21.533782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 357/697] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:21.539609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:21.703262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52ed2-8507-4f46-99ac-b7e1cf2fd170 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:21.738408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea83cc5-f8f4-44d7-ab10-4fdc2e2051a0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:21.748892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:21.749572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.749834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.750107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.751376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:21.751376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:21.753487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:21.766049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:21.768594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 358/698] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:21.777359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:21.805355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:21.807127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.807553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.807963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.816863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:21.845401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:21.847483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dc351d5d-8714-40ee-850d-24da2059b992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:21.892548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:21.893490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:21.894381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:21.894381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:21.894381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:21.894381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:21.896757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 341/699] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:21.898608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad67037-6f5a-442e-83fd-ec9ff2c3b73d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.899127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad67037-6f5a-442e-83fd-ec9ff2c3b73d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.899584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad67037-6f5a-442e-83fd-ec9ff2c3b73d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.901580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 342/700] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:21.908071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad67037-6f5a-442e-83fd-ec9ff2c3b73d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:21.952759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff49b3-0b19-4c52-b0d3-e14f9939d374 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.009922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.012603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.013188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.013746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.015042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=04b5081e9bf640679c72beb1b18f3e78) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:22.015568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.017967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e854f-0b66-4293-ae7a-a9ad6594e301 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.029309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 359/701] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:21 2022] GET /identity/v3/role_assignments?user.id=04b5081e9bf640679c72beb1b18f3e78&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:22.031267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.032382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.032868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.048501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad67037-6f5a-442e-83fd-ec9ff2c3b73d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.077303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:22.078582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:22.100253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.101014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.101296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.101629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.102400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:22.102708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.105156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.122472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 343/702] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:22.137460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:22.179815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:22.235087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 360/703] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:22.270784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7726a2-b647-4f0a-b606-61cd436720a9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.288565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:22.309312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:22.319046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.320461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.322595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.322979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.339714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.341255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:22.359352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:22.360802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:22.362226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 344/704] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:22.371318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:22.444582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7c11-548b-45c2-b45d-68e6a870ad84 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.487867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.488941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.489239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.489540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] PATH_INFO: `/v3/services/network` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.490285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.491740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-74538c28-738f-41bb-b9ef-77fc03fa016f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:22.508724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:22.509931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:22.510835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:22.511577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 361/705] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:22.512893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256600ab-e65b-4e4a-99ad-cb1bacbc7942 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.513356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256600ab-e65b-4e4a-99ad-cb1bacbc7942 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.513635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:22.513943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256600ab-e65b-4e4a-99ad-cb1bacbc7942 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.519781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 362/706] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:22.521172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256600ab-e65b-4e4a-99ad-cb1bacbc7942 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:22.535687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b395a-db3a-486e-8890-f9a9622125aa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.590585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.591420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.595544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.595544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.595544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:22.595544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.595544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55bf16-ed6c-4c4c-8954-d68bf7305161 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.611263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 345/707] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:22.613498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.613894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.614272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.636729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:22.638893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:22.655195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:22.716843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256600ab-e65b-4e4a-99ad-cb1bacbc7942 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.740078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 346/708] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:22.740078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:22.808817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.810359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.810852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.811344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.813067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] RBAC: Adding query filter params (name=network) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:22.813954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.817427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.834801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 363/709] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:22.834801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:22.847445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:22.892486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a44e8-e2b7-4c88-b6e3-0e67eb30e1aa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:22.941818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:22.942967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:22.943521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:22.944668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:22.947281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=793065092530407983e5da2446d2e939) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:22.947898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:22.950380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:22.963064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 347/710] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/role_assignments?user.id=793065092530407983e5da2446d2e939&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:22.964640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:23.030517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83c7ac-95c4-459b-8f7c-55dc155b6610 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:23.118275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ba385-3097-4940-8f51-bc4841c4fda5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:23.186022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:23.187664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.188475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.189239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.189677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:23.191203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.191668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.192040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] RBAC: Adding query filter params (type=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:23.192247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.193036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:23.196204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:23.213952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:23.215572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 348/711] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:23.260392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:23.263015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6cec9347-f4a1-4be5-8aef-964b0555a8fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:23.298441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:23.300291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:23.301965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:23.301965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:23.301965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:23.301965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:23.301965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 364/712] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:23.310439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:23.344474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:23.358084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dfa87-cea6-4334-8f27-e7d7c9117495 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.358429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dfa87-cea6-4334-8f27-e7d7c9117495 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.358651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dfa87-cea6-4334-8f27-e7d7c9117495 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.360367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 365/713] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:23.367272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.367612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.367821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.404492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:23.405943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:23.480040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a72c7-7026-4dff-bfa0-6acf5b64e51f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:23.544532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:23.545703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.546238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.546719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.547843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:23.548475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:23.550251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:23.550800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 366/714] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:15:23.574662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 349/715] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:23.576105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:23.580602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:23.703758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410405e-0570-4ec1-9ed9-93a647e887d7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:23.736385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:23.752465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:23.786086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:23.788099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.788781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.790065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.792713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:23.794684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:23.796782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297a7c3-8584-45f1-9a29-5772a7d3a801 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:23.823765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:23.825137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:23.825988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:23.833755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 367/716] 10.222.0.30 () {56 vars in 1129 bytes} [Thu Jun 16 18:15:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:23.838782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:23.867529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:23.868268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:23.868562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:23.868874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:23.869646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:23.871092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:23.906884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 350/717] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:15:23 2022] POST /identity/v3/users => generated 254 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:24.008519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057770c3-0a5c-4fc4-b917-3e755c19ddb1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:24.055500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:24.057254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:24.057747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:24.058235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:24.059317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:24.059694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:24.062451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c18aa-c3fe-44c9-8249-960fdcca4eff None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:24.070721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 368/718] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Jun 16 18:15:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:24.076779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:24.233263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5a997-a3b9-4e9d-ade6-c5d7e0818f73 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:24.269774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:24.286234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:24.286915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:24.287199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:24.287472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:24.288215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:24.288523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:24.290027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd8550-2bfd-4ecf-8466-d5ecb14147ad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:24.297878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 351/719] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Jun 16 18:15:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:24.823690 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bd900-4371-4351-9caa-9ece783ce10d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:24.824416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bd900-4371-4351-9caa-9ece783ce10d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:24.825333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bd900-4371-4351-9caa-9ece783ce10d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:24.827238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 369/720] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:24.833094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:24.835242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:24.836222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:24.881494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:24.883080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:24.985560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 352/721] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:25.045297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bd900-4371-4351-9caa-9ece783ce10d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:25.109160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:25.183928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bd900-4371-4351-9caa-9ece783ce10d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:25.226416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:25.227241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:25.227529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:25.228361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:25.247525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:25.255526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:25.256541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:25.257373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:25.261166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 370/722] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:25.266783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:25.397836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc577eb-bcac-42f7-84b4-5bc2aa0a58c1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:25.425186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:25.448810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:25.449901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:25.450385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:25.450856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:25.451873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:25.452433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:25.454237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fd7b9-40e8-43ed-99f4-9bc066a3ce5b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:25.470725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 353/723] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:25.475889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:25.623535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df5928-c418-41cd-a98a-9a3969138f98 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:25.632304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:25.667207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:25.668003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:25.668323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:25.668651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:25.681272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:25.683171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:25.687850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db73b9-b554-4257-94a2-61e6d8720d07 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:25.688219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db73b9-b554-4257-94a2-61e6d8720d07 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:25.688473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db73b9-b554-4257-94a2-61e6d8720d07 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:25.690001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 354/724] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:25.695595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:25.696191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:25.696742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:25.702435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:25.703480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:25.704802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:25.704802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:25.704802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:25.704802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:25.704802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 371/725] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:25.713785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:25.718153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:25.719715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:25.851264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 355/726] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:15:25.897698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835764e-9de0-457e-beed-f9f6e5e1d72b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:25.920528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:25.968948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:25.970640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:25.972384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:25.973010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:25.974005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:25.975728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:25.980689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.014415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 372/727] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:26.025162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.054585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abb1bb-1fde-4b71-9075-1ac87f1704e2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.073839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.087245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.136536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8962a1-406e-4501-b5e6-e3c9c64d6917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.137562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8962a1-406e-4501-b5e6-e3c9c64d6917 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.137843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8962a1-406e-4501-b5e6-e3c9c64d6917 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.138299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8962a1-406e-4501-b5e6-e3c9c64d6917 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.155127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8962a1-406e-4501-b5e6-e3c9c64d6917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2f8962a1-406e-4501-b5e6-e3c9c64d6917 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:26.163030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:26.164037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:26.166422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:26.167109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 356/728] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:26.169608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77a896-f47e-4c66-8cb5-e3d52bcce0b2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.169903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77a896-f47e-4c66-8cb5-e3d52bcce0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.170180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77a896-f47e-4c66-8cb5-e3d52bcce0b2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.171855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 357/729] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:26.179147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77a896-f47e-4c66-8cb5-e3d52bcce0b2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.219895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30176ba5-1dea-4e6c-ac86-3e819d37135e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.271798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.272772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.273125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.273466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.285490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.287416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.303210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77a896-f47e-4c66-8cb5-e3d52bcce0b2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-fd0439a0-65f6-42e7-aea3-f8f78f9dde34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:26.308851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:26.309812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:26.310779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 373/730] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:26.316222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.316222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.316222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.330747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:26.332879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:26.359018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.360305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.360598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.360883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.361813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:26.362325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.364084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.377638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 358/731] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:26.378809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.420593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.439678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 374/732] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:26.441400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.456340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.507033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e551b4f-a8a6-4fd9-b41a-c0e4043f4321 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.559566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.560840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.561122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.561395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.562247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:26.562560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.563992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.575371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b5dba-ba87-4c1f-8265-8dbbad9dfe5c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.589097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 359/733] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Jun 16 18:15:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:26.590204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.624891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.626113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.626927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.627429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.648446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.651685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:26.678468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:26.679464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:26.680388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:26.680388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:26.680388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:26.680388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:26.684414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 375/734] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:26.685330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.701805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.716157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947220c-2669-4a58-bcc1-40d6750e1070 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.752461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.765480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.767345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.767858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.768376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.769335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.770941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.792525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 360/735] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:15:26 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:26.794242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.804406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e8291-cf23-4953-b2f9-ffa80181a41e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.862902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5ab39-5ae4-4301-aa25-dba2607b92ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.863736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ab39-5ae4-4301-aa25-dba2607b92ec None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.864198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ab39-5ae4-4301-aa25-dba2607b92ec None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.864520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5ab39-5ae4-4301-aa25-dba2607b92ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.935332 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b1d77-6276-43c6-b5a2-b0ca43099e5d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.953170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5ab39-5ae4-4301-aa25-dba2607b92ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.955596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5ab39-5ae4-4301-aa25-dba2607b92ec None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.978677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.979566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.979916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.980299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.981147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:26.981523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.983021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:26.998618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 361/736] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:27.004089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.020953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.026815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.080118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 376/737] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:26 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:15:27.081927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eaccb-acf6-4e5d-9382-6582e24cba81 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.082485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eaccb-acf6-4e5d-9382-6582e24cba81 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.083030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eaccb-acf6-4e5d-9382-6582e24cba81 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.085216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 377/738] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:27.086954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eaccb-acf6-4e5d-9382-6582e24cba81 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.133292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78b583-b993-4b2d-981b-9c185b497aad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.175500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.176393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.176681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.176962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.186122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.187722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6a9eac6e-8ca5-4b83-be55-f350fe2edb8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:27.207841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:27.208892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:27.210338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 362/739] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:27.211184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53995-e468-4baa-84eb-ae5fbf0df1b5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.213630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53995-e468-4baa-84eb-ae5fbf0df1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.213630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53995-e468-4baa-84eb-ae5fbf0df1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.228634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eaccb-acf6-4e5d-9382-6582e24cba81 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.236290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e4c53995-e468-4baa-84eb-ae5fbf0df1b5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:27.237496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e4c53995-e468-4baa-84eb-ae5fbf0df1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:27.268505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.269307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.269589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.269866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.281629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.283095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:27.302457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:27.303590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:27.304536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:27.304536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:27.304536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:27.304536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:27.304536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 378/740] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:27.305336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.316855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.326285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 363/741] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:27 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:27.328271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f50ccc-9569-4609-975a-c89778c6edb9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.328656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f50ccc-9569-4609-975a-c89778c6edb9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.329034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f50ccc-9569-4609-975a-c89778c6edb9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.330693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 364/742] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:27.331219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f50ccc-9569-4609-975a-c89778c6edb9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.340750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.439897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0170e59-4e56-4963-b58e-b7e7b5e1d961 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.456164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f50ccc-9569-4609-975a-c89778c6edb9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.479266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.480022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.480335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.480609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.481457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:27.481775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.483208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c643f6-b350-4aaa-a47d-7023f1869a22 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.495245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.495919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.496233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.496505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.497229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:27.497585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.498924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.506955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 379/743] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:27.506955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.506955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.506955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.522976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:27.529295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 365/744] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:27.529450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:27.529550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.595906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.603142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.616096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 380/745] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:27 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:15:27.616096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.651443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcbcb5-1bc2-44e6-b197-168b9b11969d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.699561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.701199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.701503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.701788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.702557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.705080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.732166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 366/746] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:15:27 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:27.732166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.732440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e53bb8-14c6-407d-9f06-16919744cbf5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.783491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.784428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.784710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.784978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.799099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.800912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:27.821618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:27.824191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:27.825618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:27.825618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:27.825618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:27.825618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:27.825618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 381/747] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:27.825618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.882454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.891145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834519b4-9e0f-4cdc-ace5-622814db5a3e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.926361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.943552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.944844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.945213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.945624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.955601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.957186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.966231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041299e-8c7d-45d5-ac81-66e2103123ce None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:27.974619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:27.975849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:27.976824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 367/748] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:27.977604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.008318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.009091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.010105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.010826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.024095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:28.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:28.035984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:28.036888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 382/749] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:28.037647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.139815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae0e4-9015-44d5-b285-4b389c15eed3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.195748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.195984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301859c5-0465-42a6-9237-ab1aa306297d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.196478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.196891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.197260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.198101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:28.198534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.200300 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.202219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.217580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 368/750] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:28.218728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.255658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.256635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.256958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.257631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.258794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:28.259582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.261104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.271620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.279607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 383/751] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:28.281076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.340224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6fb8b-0545-46c6-82a1-046c83c1ac2e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.377589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.378439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.378815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.379188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.380058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:28.380840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.382341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.397617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 369/752] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:28.397617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.399426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333a1ef-8666-44fa-b97d-f5df3c989d11 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.445657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.445657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.445657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.445657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.454057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.456201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:28.473005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:28.474013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:28.475164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:28.477822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 384/753] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:28.478870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.516121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557f3d7-efe9-4a1f-baa3-f9e9cf6e9277 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.518661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.528032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.559664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.560468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.560813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.561153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.562063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=04b5081e9bf640679c72beb1b18f3e78) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:28.562467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.564171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.576266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 370/754] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/role_assignments?user.id=04b5081e9bf640679c72beb1b18f3e78&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:28.576266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.682244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df7933-d505-492e-841e-d2a133b3645c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.767073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.769008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.769008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.769008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.789386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.791656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:28.814008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:28.814970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:28.815864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:28.815864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:28.815864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:28.815864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:28.815864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 385/755] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:28.817737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df3781-875d-4f15-b645-b4da9e3528d5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.822307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.924824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.927741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.928279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.931910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.933139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:28.933655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.936320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.941470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.965278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 371/756] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:28.972794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.007305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88889b-ca94-4c0c-9e92-ce3e9fdc0ec1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.010228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:29.086487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.087995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.088465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.088775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.089645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:29.090038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.091836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:29.112285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5f83-6d11-440b-9947-1604b3d206f6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.125281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 386/757] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:29.130434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.153708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.154995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.155500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.156269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/793065092530407983e5da2446d2e939/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.158582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=793065092530407983e5da2446d2e939, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.160328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:29.176232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 372/758] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:28 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/793065092530407983e5da2446d2e939/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:29.282281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64177858-c292-4bcb-8aa3-c1307bb3eafd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.330871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.332214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.332771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.333272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.339675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:29.349459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.351631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7f862898-649c-4b5a-a3e8-3bdce6d4560d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:29.372216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:29.373288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:29.374217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:29.377208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 387/759] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:15:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:29.385561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.408605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f254ec-c398-4b6f-93c5-d04de367184c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.409796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f254ec-c398-4b6f-93c5-d04de367184c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.410291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f254ec-c398-4b6f-93c5-d04de367184c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.412698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 388/760] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:29.430721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.431395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.432188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.453045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:29.454617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:29.547885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 389/761] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:29.610607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.613714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:29.635081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9716ad7-3313-4996-9c47-fbb29d1bad20 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.689884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.690578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.690857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.691129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.691979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:29.692311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.693759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:29.734425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 373/762] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Jun 16 18:15:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:29.740594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.798164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8f90-d970-4285-901b-a89b8cb30a01 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.833160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:29.860911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.861967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.862409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.862825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.863641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.865228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:29.888966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145c286-74cf-49e9-bf30-e42da965736c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:29.900332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:29.902508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:29.904449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:29.908638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 390/763] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:29 2022] GET /identity/v3/services/network => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:29.917445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.944827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87356811-ea87-47b5-aefd-1edd40de58d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.945615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87356811-ea87-47b5-aefd-1edd40de58d0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.945966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87356811-ea87-47b5-aefd-1edd40de58d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.946308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87356811-ea87-47b5-aefd-1edd40de58d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.982615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.040825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87356811-ea87-47b5-aefd-1edd40de58d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.042533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87356811-ea87-47b5-aefd-1edd40de58d0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.174499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 374/764] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:15:30.175905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36a23-fa8f-42f9-8963-328398b3bf6c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.176245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36a23-fa8f-42f9-8963-328398b3bf6c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.176531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36a23-fa8f-42f9-8963-328398b3bf6c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.178111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 375/765] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:30.178843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf36a23-fa8f-42f9-8963-328398b3bf6c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.179742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faae4f-43ff-4461-8fdc-a0c011e72ce2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.226542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.227224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.227513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.227913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.229345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] RBAC: Adding query filter params (name=network) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:30.229660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.231071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc356-c83a-417d-a544-db3de91c85ba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.238134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.240319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 391/766] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:30.242526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.242822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.243108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.264164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:30.265430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:30.319250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf36a23-fa8f-42f9-8963-328398b3bf6c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.327227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.364726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 392/767] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:30.366244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.375063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.375945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.376276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.376603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.405080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.408237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:30.438089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:30.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:30.439981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:30.439981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:30.439981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:30.439981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:30.442650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 376/768] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:30.442650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.505750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ef4e9-4da3-4fd9-9995-b6b5f8b3e272 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.576928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.577622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.577920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.578198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.578895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] RBAC: Adding query filter params (type=network) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:30.579206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.580654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.592169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 393/769] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:30.593576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.631793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.638939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7f821-2853-420a-b1f0-b7327aaea6e2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.652601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.700448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.701587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.702016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.702455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.703352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.705174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.718043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c5246-8443-4e91-88d2-b9be5fd0099b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:30.722250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:30.723245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:30.724117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:30.727118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 377/770] 10.222.0.30 () {56 vars in 1129 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:30.728994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.766543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.767219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.767504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.767780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.768681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:30.769003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.770575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.792203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 394/771] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:30.792203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.861624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf05f-9f7a-4552-a863-79f1f583e914 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.899774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750b537-db7e-4e89-b430-04889777740c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.901413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750b537-db7e-4e89-b430-04889777740c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.902073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750b537-db7e-4e89-b430-04889777740c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.902626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750b537-db7e-4e89-b430-04889777740c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.903835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750b537-db7e-4e89-b430-04889777740c None admin] RBAC: Adding query filter params (interface=public, service_id=a22408d1b07b40d8b8b1d598a6e6b760, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:30.904399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750b537-db7e-4e89-b430-04889777740c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.906527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750b537-db7e-4e89-b430-04889777740c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.912185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780370c8-f999-419c-ac36-d49b120b3561 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.918180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 378/772] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/endpoints?service_id=a22408d1b07b40d8b8b1d598a6e6b760&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:30.919331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750b537-db7e-4e89-b430-04889777740c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.927301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.951785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.959559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.960290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.960579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.960949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.961841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:30.962192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.963862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.974006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 395/773] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:30.975010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.032741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750b537-db7e-4e89-b430-04889777740c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.069267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.070443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.070924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.071390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.079853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.081640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.091799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc0e9d-a5c7-4b78-9e71-7d7b37631485 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:31.104941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:31.106294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:31.107483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:31.115970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 379/774] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:31.117227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.131547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.132037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.132368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.132648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.133372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.134772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.141229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 396/775] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:15:30 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:31.142055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.221038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:31.223830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:31.233796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687cdec-5724-4d03-8269-3effbc543c7b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.311488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.312213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.312497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.312772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.313516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:31.313823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.315251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.328233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 380/776] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:31.335344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.389937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc30ce7-eae0-48e8-bdef-fef7e47a6316 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.441922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.442693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.442978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.443258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.444228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:31.444523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.446172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.459971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 397/777] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:31.466306 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.468103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfb135-4f00-49d7-8498-da7f9e36a9fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.515002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.515002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.515002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.515002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.515394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] RBAC: Adding query filter params (interface=public, service_id=34ccffdfe40648ad8ddffd786e325ec0, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:31.515394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.516553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a32b6-3e3a-4f87-b690-0a747efc7cb4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.522497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 381/778] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/endpoints?service_id=34ccffdfe40648ad8ddffd786e325ec0&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:31.523673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5437-705e-4bd1-8723-e4f186c4059e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.525751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5437-705e-4bd1-8723-e4f186c4059e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.525751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5437-705e-4bd1-8723-e4f186c4059e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.530239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 382/779] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:31.533622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.534150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.534668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.553805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:31.555394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:31.566440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:31.580415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:31.586845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241cfd1-3eaf-4e76-8f71-8e5a39059ecc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.630135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.630859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.631143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.631420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.632771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=93c4664744bf4bfbbc5436c5866fcb92) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:31.633492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.636190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.648083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 383/780] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:31.648083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.656777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 398/781] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/role_assignments?user.id=93c4664744bf4bfbbc5436c5866fcb92&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:31.709702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.861302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c26c-7004-4d2e-a0cc-0ee105cce020 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.897457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9129ca-a27e-46b4-9336-ad020eacea94 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.910760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49806e6-ea61-4e25-8338-3584f31c3485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.911455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49806e6-ea61-4e25-8338-3584f31c3485 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.911744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49806e6-ea61-4e25-8338-3584f31c3485 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.912024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49806e6-ea61-4e25-8338-3584f31c3485 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.912846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49806e6-ea61-4e25-8338-3584f31c3485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.914412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49806e6-ea61-4e25-8338-3584f31c3485 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.922284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 384/782] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:31.924528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62e9d-d9f8-4239-81fa-7ca21a0ede4c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.925143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62e9d-d9f8-4239-81fa-7ca21a0ede4c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.925687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62e9d-d9f8-4239-81fa-7ca21a0ede4c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.932242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 385/783] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:31.935108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.935998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.936774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.943494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.945411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.945700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.945974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.958404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:31.959958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:31.960499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:31.963734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:31.982374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:31.983606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:31.984604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:31.989907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 399/784] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:31.995585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.041807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:32.128903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 386/785] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:32.157239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.198681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ec01-0fbc-4313-8902-b634fe930104 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.251318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.252085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.252487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.252766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.253526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:32.256335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.258082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.279679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 400/786] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:32.285467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.326630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33697021-8eab-46aa-bc69-a500b5534ff5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.407480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.408865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.409148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.409446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.436425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:32.439347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:32.443727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:32.449407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:32.450361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:32.451333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:32.458691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 387/787] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:32.461056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.482934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7e5f2-4fba-4f04-8b69-cf0f3b316448 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.554234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.555055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.555340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.555613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.578845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.580442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.583768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9480-8e37-423c-86b8-b566be979acf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:32.610583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:32.611588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:32.612546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:32.612546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:32.612546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:32.612546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:32.620513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 401/788] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:32.625887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.626206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.627126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.627620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.628111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.629102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:32.629591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.631156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.646724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 388/789] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:32.648290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.721870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:32.764288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ca40e-c435-4cd7-8441-543015651979 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.812150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.814393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.816699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.817192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.835581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.838452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:32.860986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:32.862034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:32.862917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:32.862917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:32.862917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:32.862917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:32.871904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 389/790] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:32.874110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.912750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f746a-c078-4127-b8fc-6d52ee9aa505 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.996353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.997278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.997745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.999129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.000031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:33.000729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.002578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.004577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:33.023810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 402/791] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:33.029645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.090533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c475-a328-44ad-ac31-a87681ab6bbd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.116450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:33.149138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.150341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.151039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.151039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.151608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:33.151917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.153350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.198659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 390/792] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:33.212628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.224528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8489-6583-43cc-91cc-46f819b9f0fb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.276792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.278442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.278931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.279447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.295198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.304965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:33.335750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:33.336768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:33.338884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:33.345200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 403/793] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:33.352074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.411273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7765a-b740-47f3-ab8c-fe66d4917b98 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.460714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:33.486565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.488102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.488596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.489160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.523013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.524545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:33.554823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:33.555823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:15:33.558316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:33.559388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 391/794] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:33.569098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.575125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022f0a-1226-4612-9ba1-8e5d0ea2e584 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.616546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:33.627738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.628503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.628802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.629071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.629958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:33.630275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.631710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.663285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 404/795] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:33.680461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.732789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e62546-e5ae-43c7-bac3-398c13470ef4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.812402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.813015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.813295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.813640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.814500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:33.815682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.817206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.859962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 392/796] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:33.868961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.876638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2763d-f81b-4df6-a8b0-e3f501dd7bf2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.901757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:33.955655 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.956556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.956846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.957124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.984616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.986088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.997447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.014246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89e15-d4bf-44ea-9527-f1945c3bf069 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-2d18477e-d101-4687-8fed-805d4cb78e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:34.016105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:34.017180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:34.018124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:34.018124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:34.018124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:34.018124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:34.020007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 405/797] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:34.021565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c632d-63de-430a-953b-86e1bd8e70b1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.022002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c632d-63de-430a-953b-86e1bd8e70b1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.022423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c632d-63de-430a-953b-86e1bd8e70b1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.029141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 406/798] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:34.029141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c632d-63de-430a-953b-86e1bd8e70b1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.059187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336539b-69dc-4cbc-975b-e3b18f47a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.059946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336539b-69dc-4cbc-975b-e3b18f47a2dc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.060211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336539b-69dc-4cbc-975b-e3b18f47a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.060492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336539b-69dc-4cbc-975b-e3b18f47a2dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.142025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336539b-69dc-4cbc-975b-e3b18f47a2dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.143618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336539b-69dc-4cbc-975b-e3b18f47a2dc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.175868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c632d-63de-430a-953b-86e1bd8e70b1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.214007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.214677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.214967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.215227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.215952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:34.216279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.217676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45868426-0b1a-484e-a8a9-9274c2d65d49 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.235987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 407/799] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:34.235987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.235987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.235987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.248659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.253087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:34.254077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:34.266419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 393/800] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:15:34.266419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c384df5-08b0-4a01-8fdb-ce1037fe3afb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.266419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c384df5-08b0-4a01-8fdb-ce1037fe3afb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.266419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c384df5-08b0-4a01-8fdb-ce1037fe3afb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.266419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 394/801] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:34.268850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c384df5-08b0-4a01-8fdb-ce1037fe3afb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.283531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.339362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 408/802] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:34.340103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.395518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c384df5-08b0-4a01-8fdb-ce1037fe3afb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.434675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.435941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.436384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.436799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.446367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.448713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277317c-f3f5-4409-8a4e-9142b80b5f60 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b7e81543-0b01-4907-ba05-1eeae340b342 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:34.462799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:34.463843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:34.464762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 395/803] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:34.465701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.465701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.465945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.490679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:34.491023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.491899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:34.492032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.492350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.492686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.508283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.511057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:34.529974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:34.531201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:34.532130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:34.532130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:34.532130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:34.532130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:34.532130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 409/804] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:34.532418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.546060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.561950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.588944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 396/805] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:34.590064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.662727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b8e17-904a-43ec-a1d0-d9f88795ab89 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.697529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.698342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.698627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.698903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] PATH_INFO: `/v3/services/network` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.699564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.700991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.701098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47ee0-bba7-4e27-b3fb-73d0efb6146f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:34.713514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:34.714745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:34.715582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 410/806] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:34.716748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.742654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.743329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.744213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.744499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.745369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:34.745666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.747112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.761213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 397/807] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:34.762267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.793440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.806854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:34.830376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab588b8c-8003-4275-a8e1-c8f7ee719727 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.864707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.865911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.866275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.866559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.867290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:34.867601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:34.869433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc7575-072c-4306-bb2a-8e501fa873a5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:34.869574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f9cb7-8b16-48b5-b0f0-78ebe6c1083b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:34.882096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 411/808] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:34.883487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ccbd6-7890-4628-b01d-1bfe58a687c9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.883780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ccbd6-7890-4628-b01d-1bfe58a687c9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.884122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ccbd6-7890-4628-b01d-1bfe58a687c9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.885536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 412/809] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:34.886125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ccbd6-7890-4628-b01d-1bfe58a687c9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.904299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:34.905127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:34.905405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:34.905676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:34.917769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:34.925436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:34.926571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:34.927671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:34.931924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 398/810] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:34.932822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:34.997555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ccbd6-7890-4628-b01d-1bfe58a687c9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.027770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.031751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.034880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68834ff0-13af-4e5a-bb81-79cd4ce8a3e9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.035013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.035568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.035851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.036154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.036877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] RBAC: Adding query filter params (name=network) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:35.037184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.038568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.043454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 413/811] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:35.044264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.072721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b899a0-548f-49fb-8e0b-6c3e4b23f3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.073629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b899a0-548f-49fb-8e0b-6c3e4b23f3f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.073916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b899a0-548f-49fb-8e0b-6c3e4b23f3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.074187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b899a0-548f-49fb-8e0b-6c3e4b23f3f3 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/04b5081e9bf640679c72beb1b18f3e78/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.076210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b899a0-548f-49fb-8e0b-6c3e4b23f3f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=04b5081e9bf640679c72beb1b18f3e78, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.077645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b899a0-548f-49fb-8e0b-6c3e4b23f3f3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.091845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 399/812] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:34 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/04b5081e9bf640679c72beb1b18f3e78/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:35.093351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.093643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.093985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.115064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:35.116370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:35.174555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc172c0-68df-4690-aa76-f6a1172b2ccd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.216073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 400/813] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:35.216073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.227091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.228237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.228602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.228961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.239043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.240650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:35.258966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:35.263016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 414/814] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:35.264081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.324607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.336766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.342650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8f97d-1f01-4ac3-86f6-8877b366b3cc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.378632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.379312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.379588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.379863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.380756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:35.381063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.382629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.391297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 401/815] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:35.392163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.402002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56a859-c1f4-4f54-b280-9abc48935515 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.442221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.442917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.443197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.443474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.444274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:35.444582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.446166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.456906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 415/816] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:35.456906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.514397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eab6a4-02ee-41bd-a7bd-cc0edab743b4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.551704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.552663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.553018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.553367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.554127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.555574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:35.568553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:35.569729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:15:35.570730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:35.572409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 402/817] 10.222.0.30 () {56 vars in 1129 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:35.573402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.585335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64593-46be-4f1d-8e86-de558df804c4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.593202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.606203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.625340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.626008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.626287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.626554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.627344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:35.627652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.629103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.647865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 416/818] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:35.647865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.699678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e24c13-8a9d-4915-adf4-0452e8a9e727 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.738991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.740564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.740990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.741403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.753853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.755555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.759273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c1594-b7e9-4919-a4f2-35d7babd64e1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:35.774334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:35.776069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:35.776993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:35.776993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:35.776993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:35.776993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:35.776993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 403/819] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:35.776993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.803616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.804676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.805143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.805422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.806933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.808348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.840082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 417/820] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:15:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:35.840082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:35.869936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.893120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:35.933717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a540393-df1e-41d2-8a34-596ab3c5b8a1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.984797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:35.985487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:35.985774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:35.986053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:35.986778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:35.987088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:35.988766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:35.989343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a77a4-b9e6-474b-bf87-6a86aee6fc45 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:35.996193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 404/821] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:35.997089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:36.049763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.051127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.051615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.052205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.053833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=793065092530407983e5da2446d2e939) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:36.054398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.057724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.070081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 418/822] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/role_assignments?user.id=793065092530407983e5da2446d2e939&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:36.074846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:36.145750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b174c-e218-41e7-b46e-dbd4373dfcd4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:36.195849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:36.208157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.209330 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.209827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.210296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.211656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:36.215562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.217215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.233966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 405/823] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:36.244622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:36.244809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:36.255772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df346e9-fd21-4f0a-a666-2592b86c8a3b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:36.316189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.316885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.317168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.317444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.318960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:36.319273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.321069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.328212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 419/824] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Jun 16 18:15:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:36.333548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:36.390709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26d163-326b-4171-ab9b-56630bb6eb02 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:36.444472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.445321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.445603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.445999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.455743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.457298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:36.475660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:36.477021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:36.478814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:36.482393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 406/825] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:15:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:36.483835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:36.495667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4106c7-25ac-4a8f-8ff4-44a135a39a27 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:36.559104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.560383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.560832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.561261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.562896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.564842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:36.565679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.600493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 420/826] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:15:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:36.648302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ba9e8-c39c-463c-a1ae-790adec55d19 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:36.699752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.700841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.701278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.701701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.702704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:36.703192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.704883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.728377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 407/827] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Jun 16 18:15:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:36.736896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:36.756449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:36.867295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf3821-bd2a-4465-8a09-e9059f5cd8f6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:36.905271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:36.906724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:36.907205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:36.907673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:36.920259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:36.922095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:36.941934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:36.942888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:36.943774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:36.943774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:36.943774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:36.943774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:36.950677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 421/828] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:36.953162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.063131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5ea52-d3ff-418e-a594-0519aa4600e9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:37.064628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:37.101365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:37.102051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.102334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.102610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.103970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:37.104308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:37.105715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:37.119314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 408/829] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:37.125079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.199143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:37.238575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236c4d-175a-4488-9127-2f1fa3a5ed1e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:37.278874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:37.280184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.280696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.281424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.295218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:37.296845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-48801f81-cf37-4a5e-b8cc-8011caecc09a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:37.312649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:37.313651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:37.315039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 422/830] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:37.322357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.345000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9e23b-c060-4472-a609-9f6b93c066a1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.345477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9e23b-c060-4472-a609-9f6b93c066a1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.345766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9e23b-c060-4472-a609-9f6b93c066a1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.347396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 423/831] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:37.352272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.352559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.352945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.375463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:37.376735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:37.446753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3fc69-18eb-4fec-8800-aa3035ec412e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:37.476913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 424/832] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:37.493139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:37.494834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.495495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.496036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.497078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:37.497599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:37.499417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:37.500198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.528320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 409/833] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:37.530031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.559901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:37.561932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:37.612941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7632c7-d5b4-4d99-a919-0ddead2064da None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:37.650207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfa58-5572-4138-a6e8-72237520e7b9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:37.655458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:37.655887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.656550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.656831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.669154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:37.677999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:37.679151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:37.680499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 425/834] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:37.685271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.707809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:37.709018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.709301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.710109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/93c4664744bf4bfbbc5436c5866fcb92/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.727072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=93c4664744bf4bfbbc5436c5866fcb92, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:37.730320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:37.756362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 410/835] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:37 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/93c4664744bf4bfbbc5436c5866fcb92/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:37.810280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac6d02-86f9-49c7-a983-1f2a1262e508 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:37.829741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:37.850691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:37.851487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:37.851774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:37.852073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:37.853011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:37.853475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:37.854914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:37.864930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 426/836] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:37.870553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:37.968258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:38.000625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42b70a-11d3-476d-afa4-9d534436c7bf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:38.037114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:38.037921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.038201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.038473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.050098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:38.051546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:38.070879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:38.071913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:38.073054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:38.073054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:38.073054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:38.073054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:38.073054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 411/837] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:38.078513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:38.198063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321a1b-3659-4e6f-b731-77818fe824fd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:38.234668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:38.235346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.235631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.235908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.236678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:38.236988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:38.238360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:38.256401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 427/838] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:38.258089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:38.372308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76542b4-5658-4964-9b7d-36bfd35d7f6f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:38.398434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:38.410739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:38.411554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.411840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.412149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.422037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:38.423528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7807aeca-0c8d-4aef-8aa9-518cc9537299 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:38.442931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:38.443950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:38.444915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 412/839] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:15:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:38.453398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:38.499335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:38.578959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf34cb-e90d-4c9d-8cbd-d75e343a9e22 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:38.614929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b62ef-8977-4e0e-b7c4-7ce4f1854ce5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.615285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b62ef-8977-4e0e-b7c4-7ce4f1854ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.615489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b62ef-8977-4e0e-b7c4-7ce4f1854ce5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.617511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 413/840] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:38.627279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.628560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.628560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.632490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:38.633089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.633366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.633643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.635305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:38.635622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:38.637111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:38.659964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:38.662224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:38.668632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 428/841] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Jun 16 18:15:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:38.672635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:38.760190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 414/842] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:38.783136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:38.823117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:38.832218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd77bb-3b9d-44be-95bb-3956ef7b01bf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:38.878044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e448f-6745-47c5-811c-92c8b2d77a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:38.879107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e448f-6745-47c5-811c-92c8b2d77a33 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.879621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e448f-6745-47c5-811c-92c8b2d77a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.880188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e448f-6745-47c5-811c-92c8b2d77a33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.920916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e2f01-b368-49de-84c4-b0d74621eca5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:38.965634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:38.967205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:38.967798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:38.968351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:38.969266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e448f-6745-47c5-811c-92c8b2d77a33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:38.969744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:38.971193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e448f-6745-47c5-811c-92c8b2d77a33 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:38.972382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:38.988843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:38.998855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:39.000422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:39.001921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:39.009947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 415/843] 10.222.0.30 () {56 vars in 1132 bytes} [Thu Jun 16 18:15:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:39.011654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.236838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:39.238134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50138-7e3b-4e2e-af64-3cbf4ccf281d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:39.248511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 429/844] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:15:39.250235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366edb7-c1c2-4028-981e-d03017b8f423 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.250589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366edb7-c1c2-4028-981e-d03017b8f423 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.250939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366edb7-c1c2-4028-981e-d03017b8f423 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.252536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 430/845] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:39.254456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366edb7-c1c2-4028-981e-d03017b8f423 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.298218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:39.299288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.299576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.299856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.300614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:39.301062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:39.302598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f32112-9cf9-4a09-8ae7-bf4cb216675b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:39.309405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 416/846] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:39.310654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.310956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.311240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.335839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:39.337177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:39.386589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:39.410031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366edb7-c1c2-4028-981e-d03017b8f423 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:39.432121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 417/847] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:39.432121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.456780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:39.457587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.457873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.458182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.472503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:39.475235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:39.496961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:39.497928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:39.498826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:39.498826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:39.498826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:39.498826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:39.504236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 431/848] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:39.506343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.563161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9be93-992f-4805-8213-eaa37108e6da None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:39.574268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:39.614263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:39.614907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.615331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.615565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.617082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:39.617437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:39.619934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:39.626708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 418/849] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:39.627640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.659377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c97a-d091-4c27-b7c4-9534881eedda None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:39.708308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:39.709613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.710207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.710748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.711691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:39.714554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:39.730051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:39.731142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:39.731984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:39.735252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 432/850] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:39.741303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.847585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:39.878668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5f2dc-b3ac-42a2-9bbc-16ce7db34f9c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:39.893743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:39.918632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:39.919374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:39.919657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:39.920012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:39.920759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:39.921068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:39.922442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:39.928442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 433/851] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:39.935666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:39.955833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43d40-491d-4162-9748-f53651f56793 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.005652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.006886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.007492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.008114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.009471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:40.010133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:40.012750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:40.038325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 419/852] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:40.041002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:40.082127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e39d-82ba-4f9f-a62a-e32d9f8e29e1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.125520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.126273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.126690 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.127061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.127921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:40.128363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:40.129825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:40.144334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 434/853] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:40.181760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ec00d-413c-4588-9f3f-214148960905 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.205656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:40.222121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.222947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.223224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.223494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.232514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:40.234126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3151cdc0-448b-4ad9-ba69-8be5fb2e2584 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:40.251212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:40.252204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:40.253092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 420/854] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:40.260944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:40.312600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:40.344156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf4c41-7266-42b8-a517-1a8a7b02fd62 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.344705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf4c41-7266-42b8-a517-1a8a7b02fd62 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.344996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf4c41-7266-42b8-a517-1a8a7b02fd62 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.347479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 421/855] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:40.352724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.353191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.353795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.379230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:40.380856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:40.388975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd7d6-0190-41f3-b93c-3f9f8fbc9de7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.440286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.440975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.441251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.441536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.442366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:40.442682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:40.444201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:40.465699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 435/856] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:40.470470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:40.492070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 422/857] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:40.512599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:40.617820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:40.622389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459697-a247-4d54-aad5-5702c9f22a36 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.635614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:40.656923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18601040-0dd0-4417-bef1-ca15140abf42 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.675620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.677564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.678094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.678514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.679766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=04b5081e9bf640679c72beb1b18f3e78) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:40.680629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:40.682957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:40.692569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 436/858] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:40 2022] GET /identity/v3/role_assignments?user.id=04b5081e9bf640679c72beb1b18f3e78&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:40.705140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.706251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.706684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.707131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.722278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:40.731113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:40.732367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:15:40.733401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:40.740190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 423/859] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:40.742377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:40.860232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91d17e-871f-45e3-9a80-d80677345113 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:40.902554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:40.903489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:40.903926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:40.904365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:40.905253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:40.905704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:40.907489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:40.920191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 437/860] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:40.923419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:41.035344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ea9fe-f248-401c-ba0f-caf7bbf1b0da None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:41.070509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:41.071326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.072337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.072688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.083869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:41.085365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:41.094300 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:41.114242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:41.115319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:41.116430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:41.116430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:41.116430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:41.116430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:41.116430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 424/861] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:41.122751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:41.151695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:41.232401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59fe12-5cb4-47f8-9377-dfa11926b13b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:41.268147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:41.269068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.269492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.269907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.270773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:41.271248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:41.273201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:41.287730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 438/862] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:41.293933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:41.407469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0aeb11-a880-4707-8d99-214898599722 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:41.444226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:41.445297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.445721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.446137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.455654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:41.457366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:41.472962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:41.474225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:41.475408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:41.477683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 425/863] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:15:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:41.485254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:41.592497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:41.608229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eb778-c223-42b9-b1fb-19edd9ddfe8e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:41.645120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:41.646076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.646517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.646933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.647918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:41.648391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:41.650606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513ee9b-7167-4ebb-a7b6-ed4300f79f3d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:41.670429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 439/864] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Jun 16 18:15:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:41.676500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:41.740804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:41.785651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceb8eb-2002-4e60-9f45-e63f729a4888 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:41.826087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:41.826993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.827449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.827863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.906641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:41.908352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:41.925850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa78a22-af17-46ab-9137-e3acc1997575 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.926392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa78a22-af17-46ab-9137-e3acc1997575 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.926392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa78a22-af17-46ab-9137-e3acc1997575 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.928692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 440/865] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:41.939678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:41.940017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:41.940277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:41.959521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:41.961132 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:41.990708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:42.041962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 441/866] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:42.061545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 426/867] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:15:42.071205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.071438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.162638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:42.191952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e245-d9e4-4c5a-9766-a712098021b4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:42.232604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:42.233437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.233719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.233996 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.245991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:42.247477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:42.248219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2500235-5db2-424d-9987-455496d328e6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:42.266833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:42.268747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:42.270794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:42.270794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:42.270794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:42.270794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:42.272647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 442/868] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:42.273913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.308996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef6c94-ee4d-46e3-b17a-de6880fea7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:42.309593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6c94-ee4d-46e3-b17a-de6880fea7af None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.309866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6c94-ee4d-46e3-b17a-de6880fea7af None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.310150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6c94-ee4d-46e3-b17a-de6880fea7af None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.310882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef6c94-ee4d-46e3-b17a-de6880fea7af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:42.312509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef6c94-ee4d-46e3-b17a-de6880fea7af None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:42.340465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 427/869] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:15:42 2022] POST /identity/v3/services => generated 269 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:42.409454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2c5ec-0002-4a65-8cd2-84ae8eb7fbd8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:42.441403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:42.446293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f6e8-c115-49a1-ba20-359a0f5c87e9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.447865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f6e8-c115-49a1-ba20-359a0f5c87e9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.448748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f6e8-c115-49a1-ba20-359a0f5c87e9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.452356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 428/870] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:42.458095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.458998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.459954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.460457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:42.461196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.461472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.461772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.463295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:42.463631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:42.465259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:42.484174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 443/871] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:42.489019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.495279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:42.496598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:42.527417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:42.602036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 429/872] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:42 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:42.626697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.637886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f648c-f1c7-4658-bda3-31b27d7306f5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:42.683307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:42.684475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.684906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.685410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.693631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:42.695508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:42.714383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:42.715496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:42.716537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:42.723073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 444/873] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:42.726023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.772302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fe1e8-55e5-42ad-a9a7-07af8130b8ae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:42.781832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:42.819454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:42.820537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.821094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.822172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.823246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:42.824935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:42.842788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 430/874] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:15:42 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:42.866551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29498d77-3b25-4289-abf4-1b1d29fdb9d2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:42.920027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:42.921222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:42.921731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:42.922173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:42.923178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:42.923763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:42.925598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03378da7-52cf-48a5-a515-803dcdda4aed None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:42.945713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 445/875] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:42.948557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:42.980501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:43.077839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948cfd0-76c0-477a-84d0-be3a8c0970c4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:43.122012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:43.122614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:43.122898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:43.123171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:43.124022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=93c4664744bf4bfbbc5436c5866fcb92) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:43.124372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:43.125832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d3fb4-8b75-4e05-95e4-e8e537ab55f2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:43.136185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 431/876] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:42 2022] GET /identity/v3/role_assignments?user.id=93c4664744bf4bfbbc5436c5866fcb92&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:43.162546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9928-6b3b-45fd-8996-b941e7b5b7b6 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:43.162835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9928-6b3b-45fd-8996-b941e7b5b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:43.163119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9928-6b3b-45fd-8996-b941e7b5b7b6 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:43.165025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 446/877] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:43.171464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:43.172363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:43.172363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:43.198844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:43.200133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:43.303311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 432/878] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:43.332015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9928-6b3b-45fd-8996-b941e7b5b7b6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:43.490322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:43.497197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9928-6b3b-45fd-8996-b941e7b5b7b6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:43.545402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:43.546590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:43.546969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:43.547414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:43.548639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:43.550295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:43.567556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:43.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:43.570476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 447/879] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:43.583318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:43.586961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:43.748217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd4b5-694d-4ab9-8f78-cac82dd566b4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:43.796439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:43.797133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:43.797414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:43.797682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:43.798412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:43.798714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:43.800278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d5cb-5beb-49a9-90ba-252a6123f71c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:43.808076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 433/880] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:43.814046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:43.941217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e140d4-82e6-40a1-9c55-185b44797324 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:43.980717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:43.982263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:43.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:43.983236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:43.984200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:43.984717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:43.986307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99d4ad-222c-421e-bbf0-46da09311642 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:43.991992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 448/881] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:44.856647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fa82a-ce0f-4478-b945-2a606fc30faf None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:44.857186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fa82a-ce0f-4478-b945-2a606fc30faf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:44.857186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fa82a-ce0f-4478-b945-2a606fc30faf None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:44.858897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 434/882] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:44.864502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:44.864791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:44.865177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:44.883340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:44.885116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:44.909747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:44.968896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 449/883] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:44.993481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fa82a-ce0f-4478-b945-2a606fc30faf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:45.082011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:45.139677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fa82a-ce0f-4478-b945-2a606fc30faf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:45.183766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:45.184650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.184936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.185214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.185878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:45.187254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-29a162fe-7b38-4b55-a2d5-38ae96d9d7ba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:45.201262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:45.202261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:45.203103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 435/884] 10.222.0.30 () {56 vars in 1132 bytes} [Thu Jun 16 18:15:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:45.209248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:45.343122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b471c37-6aa9-4c88-94cf-84ef32649935 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.343766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b471c37-6aa9-4c88-94cf-84ef32649935 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.344284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b471c37-6aa9-4c88-94cf-84ef32649935 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.346220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 436/885] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:45.351917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.352505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.353075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.372838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:45.375440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:45.398450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80a97c-d818-4b2b-b402-0756bfec1a57 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:45.465297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:45.466548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.467116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.467702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.469116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:45.469745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:45.472477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:45.487496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 450/887] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:45.487876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 437/887] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:45.489671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:45.510115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:45.538233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:45.629017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:45.660904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee3a33-128e-432b-8897-b8c6dd668ea9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:45.685283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d41d52-ad20-4e75-9371-399fc3bd3220 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:45.731677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:45.733394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.734225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.734523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:45.734951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.735673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.736129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.736285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:45.736772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.736884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:45.737659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:45.739157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:45.739341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:45.750719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 451/888] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:45.754771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:45.755748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:45.756601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:45.757323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:45.758700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 438/889] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:45.765103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:45.880329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:45.909369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30de5-eb38-41ab-97a9-ab9f0b8a0398 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:45.962897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304357f-22a9-4f53-b274-810960151587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:45.964098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304357f-22a9-4f53-b274-810960151587 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:45.964526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304357f-22a9-4f53-b274-810960151587 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:45.964940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304357f-22a9-4f53-b274-810960151587 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:45.966583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304357f-22a9-4f53-b274-810960151587 None admin] RBAC: Adding query filter params (interface=public, service_id=eb3d1145fffa44ff8ad9a096e28816c6, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:45.967049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304357f-22a9-4f53-b274-810960151587 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:45.971710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304357f-22a9-4f53-b274-810960151587 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:45.974634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402cf23-5272-4697-a634-bdc0f30d80d8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:45.986231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 452/890] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:15:45 2022] GET /identity/v3/endpoints?service_id=eb3d1145fffa44ff8ad9a096e28816c6&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:45.995016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304357f-22a9-4f53-b274-810960151587 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:46.061491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:46.079646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:46.080523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.080635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.080899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.081632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:46.081938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:46.083305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:46.097359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 439/891] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:46.104411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:46.140931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304357f-22a9-4f53-b274-810960151587 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:46.181949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26576f11-19d1-4ff0-a1da-7ce001078ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:46.183424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26576f11-19d1-4ff0-a1da-7ce001078ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.183914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26576f11-19d1-4ff0-a1da-7ce001078ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.184458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26576f11-19d1-4ff0-a1da-7ce001078ef6 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.185509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26576f11-19d1-4ff0-a1da-7ce001078ef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:46.187251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26576f11-19d1-4ff0-a1da-7ce001078ef6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:46.204536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 453/892] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:15:45 2022] GET /identity/v3/services => generated 1268 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:46.328037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2a451-7f56-4aac-82ac-36256a1ec9bf None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.329296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2a451-7f56-4aac-82ac-36256a1ec9bf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.329780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2a451-7f56-4aac-82ac-36256a1ec9bf None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.331769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 454/893] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:46.341638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16735d60-ebc4-4f4d-8817-87f92353843e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.342620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16735d60-ebc4-4f4d-8817-87f92353843e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.343142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16735d60-ebc4-4f4d-8817-87f92353843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.368505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-16735d60-ebc4-4f4d-8817-87f92353843e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:46.371215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-16735d60-ebc4-4f4d-8817-87f92353843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:46.384491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:46.398758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d2030-6ff6-4691-932f-cc6d623eb180 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:46.474601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:46.475916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.476423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.477150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.478118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:46.478627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:46.480248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:46.482895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 455/894] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:46.488105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247387ec-a869-4606-881d-f0d854c67afc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.488105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247387ec-a869-4606-881d-f0d854c67afc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.488105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247387ec-a869-4606-881d-f0d854c67afc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.488380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 440/895] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:46.489326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 456/896] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:46.496355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:46.496890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.497398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.497894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.531621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:46.532843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:46.576903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:46.629449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2c498-441b-40d4-b424-24d8751926fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:46.646529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 457/897] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:46.647726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:46.674438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:46.675361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.675788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.676238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.677426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] RBAC: Adding query filter params (interface=public, service_id=3f828beaa95d4e09a149b58c6f9c79a7, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:46.678019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:46.679572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:46.691175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 441/898] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:15:46 2022] GET /identity/v3/endpoints?service_id=3f828beaa95d4e09a149b58c6f9c79a7&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:46.692384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:46.761874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:46.808854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e7324-1938-47ba-b572-4fd892091294 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:46.835613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1239ee-97f0-4f15-b2d8-a55e78a17339 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:46.867452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:46.869083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.869453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.869871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.870477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:46.889618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:46.899021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a99860-1b50-497e-9687-936235314500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:46.900491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a99860-1b50-497e-9687-936235314500 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:46.901043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a99860-1b50-497e-9687-936235314500 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:46.901504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a99860-1b50-497e-9687-936235314500 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:46.902913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a99860-1b50-497e-9687-936235314500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:46.905007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a99860-1b50-497e-9687-936235314500 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:46.905696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:46.906901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:46.907762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 458/899] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Jun 16 18:15:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:46.909466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:46.919586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 442/900] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:15:46 2022] GET /identity/v3/services => generated 1268 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:46.920964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a99860-1b50-497e-9687-936235314500 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:47.093642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a99860-1b50-497e-9687-936235314500 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.095343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53721cb9-e9e1-4302-9a94-5a77132bf800 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.144337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.145765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.146294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.146779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.148032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:47.149253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:47.149608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.151937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:47.159702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.160848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.161554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.161730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.163001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.166030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:47.166635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 443/901] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:15:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:47.174405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:47.195915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 459/902] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:15:46 2022] POST /identity/v3/services => generated 241 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:47.244603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:47.358710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1cd00-21d8-4789-82dc-e1a0fb0cc186 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.405148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.406463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.406895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.407313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.424091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.425409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:47.457658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:47.458620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:47.459741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:47.459741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:47.459741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:47.459741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:47.460171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 444/903] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:47.467581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:47.623208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd1568-da4f-4c41-8ff9-d49b665b0cab None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.692815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.694370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.695124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.695571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.696536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:47.697012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.698553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:47.712963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 460/904] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:47.719493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:47.896529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:47.912335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3b0a-9b5d-4538-97aa-4bb8939994c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.976356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.977663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.978676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.979340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.994646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.997317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:48.025146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:48.026286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:48.027235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:48.027772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 445/905] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:15:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:48.034630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:48.100910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:48.168089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe5a76-00ea-43ed-8394-e5b3c5475044 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:48.211024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:48.211720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.212526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.212815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.213881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:48.214191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:48.215655 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:48.236753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 461/906] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Jun 16 18:15:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:48.242601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:48.385384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95661e-a8ad-4a9a-a518-a5f4b5c1eab9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:48.450200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb6be1-1ec0-43f4-bd69-e34b9b7b5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:48.451295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb6be1-1ec0-43f4-bd69-e34b9b7b5434 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.451778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb6be1-1ec0-43f4-bd69-e34b9b7b5434 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.452295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb6be1-1ec0-43f4-bd69-e34b9b7b5434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.488536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:48.599238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb6be1-1ec0-43f4-bd69-e34b9b7b5434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:48.601511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb6be1-1ec0-43f4-bd69-e34b9b7b5434 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:48.789260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 446/907] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:15:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:15:48.797879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:48.936853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e0e5-fb53-4f90-971b-3d7d24279336 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:48.953167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:48.964536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a771d-73fb-4469-96fc-211f6bbb806d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.965199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a771d-73fb-4469-96fc-211f6bbb806d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.965683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a771d-73fb-4469-96fc-211f6bbb806d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.971335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 447/908] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:48.973482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.974011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.974502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.986692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:48.988678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.989109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.989401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.995961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:48.997558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:49.007003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.010197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:49.035171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:49.036192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:49.037101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:49.037101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:49.037101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:49.037101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:49.039822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 462/909] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:49.045991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.070929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:49.128142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 448/910] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:49.164032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.191100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debac807-4a81-4c42-a968-6131a94ec743 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.234191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.234876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.235163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.235441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.236209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:49.236519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.238207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.253983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 463/911] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:49.260625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.290593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7248b3a-ac7d-4bc8-a2a8-e7a591f632a3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.294167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:49.331388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.332702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.333201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.333681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.334614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.336572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:49.351562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:49.352899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:49.354103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 449/912] 10.222.0.30 () {56 vars in 1132 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:49.360472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.402467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e161-12a7-4e77-b115-89976716e6fd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.443871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:49.468912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.470678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.471452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.472240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.488478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.491755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:49.524037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:49.525562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:49.526775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:49.529364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 464/913] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:49.530641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ba91-b985-44bc-b682-68d144c179fb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.542825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.573640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.574367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.574657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.574942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.575659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:49.575970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.577668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.584103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 450/914] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:49.591945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.655509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:49.677141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8520c-4012-49fe-b199-2fda3f26000f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.724711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.725508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.725850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.726137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.727172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b756e5b-136b-419d-95fc-9e5e466c22d9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.727612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:49.728692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.731416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.740182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:49.748311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 465/915] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:49.757308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.779640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.780743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.781499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.781998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.783036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:49.783558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.785331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cadac-c817-4596-9e79-29099118fd2a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.792974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 451/916] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:49.794862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a96ce-f5e5-46de-b575-bc8c509fb006 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.795605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a96ce-f5e5-46de-b575-bc8c509fb006 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.795933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a96ce-f5e5-46de-b575-bc8c509fb006 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.797664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 452/917] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:49.799107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a96ce-f5e5-46de-b575-bc8c509fb006 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:49.908234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb3d98-cc3d-403f-97d5-b1067c502545 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.944916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a96ce-f5e5-46de-b575-bc8c509fb006 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:49.955462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:49.955706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.955987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.956469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.957207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] RBAC: Adding query filter params (role.id=2a17f711a8a648448b0285630ea71d9a, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=93c4664744bf4bfbbc5436c5866fcb92) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:49.957563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:49.959343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc440e9-e1e5-4e3e-a9b0-f9f37c261dae None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.976096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 466/918] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:15:49 2022] GET /identity/v3/role_assignments?user.id=93c4664744bf4bfbbc5436c5866fcb92&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=2a17f711a8a648448b0285630ea71d9a => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:49.976096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85164557-cadf-4add-8b66-0829a1662f2a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:49.976096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85164557-cadf-4add-8b66-0829a1662f2a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:49.976096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85164557-cadf-4add-8b66-0829a1662f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:49.994860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:50.002184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-85164557-cadf-4add-8b66-0829a1662f2a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:50.003555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-85164557-cadf-4add-8b66-0829a1662f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:50.015216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.016611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.017152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.017819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.019199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.021827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.060125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 453/919] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:15:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:50.104657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 467/920] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:50.131734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.171348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:50.202756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb882d-a9ac-4ca4-aef7-981e3ff301cc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.203177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb882d-a9ac-4ca4-aef7-981e3ff301cc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.203486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb882d-a9ac-4ca4-aef7-981e3ff301cc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.206451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 468/921] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:50.214998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.215446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.215885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.251580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:50.253582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:50.296536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4e717-0c25-4f15-b3fc-461160ea79b0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.357397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.358228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.358515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.358793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.359475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.361263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.375567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 469/922] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:50.385509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:50.386655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:50.387560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 454/923] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:50.393201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.413100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.470366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:50.536380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff879c-2604-4e6b-9faf-86da5c943588 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.574602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:50.594570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef95bf-b293-4c17-bf8e-00e55ee2b7ee None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.604832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.605659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.605946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.606222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.606952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:50.607393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.609916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.620090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 470/924] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:50.627092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.653007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.654150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.654626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.655047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.655861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.657832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:50.673264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:50.674546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:15:50.675561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:50.679119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 455/925] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:50.687200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.803935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d7128-3480-4590-88a3-e856e8ca2c6b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.845506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:50.861862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.863487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.864080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.864794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.866037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:50.866256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4a441-7505-4f8c-9136-0d0df3085944 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.867130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.874041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.887551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 471/926] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:50.894240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.937609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.939816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.940556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.941140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.942318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:50.943376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.945699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.953981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 456/927] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:50.959659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.003282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:51.093260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea5c80-f7cb-4841-9cc8-1e296d88f3e0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.142097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.143065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.143365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.143615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.144822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] RBAC: Adding query filter params (interface=public, service_id=cc362852efd24b619ed34e2018519d3e, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:51.145104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.146759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.160324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 472/928] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/endpoints?service_id=cc362852efd24b619ed34e2018519d3e&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:51.161355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.172555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ca764-3dcd-4850-b1fb-e6fdb8e00dc0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.233472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:51.247933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.249637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.250541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.251324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.253296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:51.254329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.257574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.269422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 457/929] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:51.279356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.349863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70c2c7-0479-4beb-b0ba-1c60a1d39df1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.420035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585dd7d-0e30-41b1-bbb9-15b8f53723d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.420941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585dd7d-0e30-41b1-bbb9-15b8f53723d1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.421453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585dd7d-0e30-41b1-bbb9-15b8f53723d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.421781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585dd7d-0e30-41b1-bbb9-15b8f53723d1 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.422619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585dd7d-0e30-41b1-bbb9-15b8f53723d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.424238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585dd7d-0e30-41b1-bbb9-15b8f53723d1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.436214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 473/930] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:15:51 2022] GET /identity/v3/services => generated 1497 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:51.455659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:51.474897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9ec61-ced1-4a53-b7b2-b5c7d9352869 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.535509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff870b-a8ac-4a3e-a847-f169115d8aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.536309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff870b-a8ac-4a3e-a847-f169115d8aec None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.536673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff870b-a8ac-4a3e-a847-f169115d8aec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.536953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff870b-a8ac-4a3e-a847-f169115d8aec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.537975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff870b-a8ac-4a3e-a847-f169115d8aec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.539527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff870b-a8ac-4a3e-a847-f169115d8aec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.575017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 458/931] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:15:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:52.679558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157332b8-7d0f-444a-b149-2bc0eb022f14 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:52.680407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157332b8-7d0f-444a-b149-2bc0eb022f14 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:52.680565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157332b8-7d0f-444a-b149-2bc0eb022f14 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:52.684473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 474/932] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:52.694151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a590f-ce9a-414d-89e3-ec224da9feca None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:52.694816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a590f-ce9a-414d-89e3-ec224da9feca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:52.695432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a590f-ce9a-414d-89e3-ec224da9feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:52.726206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-601a590f-ce9a-414d-89e3-ec224da9feca None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:52.729671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-601a590f-ce9a-414d-89e3-ec224da9feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:52.856215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 459/933] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:52.896684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157332b8-7d0f-444a-b149-2bc0eb022f14 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:52.997425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d202-e2b7-45d6-934f-829778749af8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:52.997938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d202-e2b7-45d6-934f-829778749af8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:52.998441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d202-e2b7-45d6-934f-829778749af8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.001581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 460/934] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:53.011342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.011824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.012530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.042832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:53.045123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:53.083552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157332b8-7d0f-444a-b149-2bc0eb022f14 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:53.116273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:53.147082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:53.148652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.149159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.149720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.167735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:53.179085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:15:53.192243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:53.193852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:53.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 475/936] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Jun 16 18:15:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:53.196962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 461/936] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:53.199169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:53.232940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:53.383115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1cd3c-9bcc-472a-b046-a98c8c60c481 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:53.425184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:53.426268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.426763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.427342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.428583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:53.429204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:53.431404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:53.434999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2339b-f088-40e1-94ab-bec024ff2777 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:53.442529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 476/937] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:15:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:53.454519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:53.489043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:53.491018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.491589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.492162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.493993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:53.495721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:53.516915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:53.517980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:15:53.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:53.527564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 462/938] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:53.531790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:53.554241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:53.657715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:53.678223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef743029-6fa0-4453-9633-e421732fc045 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:53.732957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b8cc3-910e-4b4d-b9a5-93628f651038 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:53.749874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:53.750905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.751205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.751500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.779646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:53.784090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:53.794125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:53.795312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.795839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.796431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.797392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:53.797911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:53.800280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:53.807624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 463/939] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:53.816799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:53.824607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:53.825793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:53.827352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:53.827352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:53.827352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:53.827352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:53.827352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 477/940] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:53.833383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.017666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813b6cb-f881-4b91-9a0e-3cfb28aeb34d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.049022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.072880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.074895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46806-f63c-4308-b010-5cd7c8f30d1f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.097174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.098536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.099119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.099752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.101229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:54.101901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.104838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79eba3-42e4-40d1-b282-56e10959495a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.120089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 464/941] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:54.141984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59b3e5-95e7-468b-a2a6-3d91aeda3ad3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.142327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59b3e5-95e7-468b-a2a6-3d91aeda3ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.142568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59b3e5-95e7-468b-a2a6-3d91aeda3ad3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.148723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 465/942] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:54.150925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.151262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.151549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.155146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.157218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.158048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.158829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.160627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:54.161507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.164551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.174334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:54.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:54.200382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 478/943] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:54.203994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.296260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 466/944] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:54.325893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.418115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884bd84-acf1-4d22-8489-02cf1257f331 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.492925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.494898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.495595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.496150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.509640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.511640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.515355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904845b-d115-4f61-bc19-d1225d0e97ec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:15:54.534273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:54.535318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:54.536288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 479/945] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:54.544534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.549747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.571224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.572898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.573340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.574283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.574899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.575831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.577956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:54.592809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:54.593909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:54.594750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:54.597743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 467/946] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:54.611423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.702486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae19d2-1206-4c8d-a2b5-40c5aa788f23 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.748187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.749954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.750780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.750780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.751351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:54.756084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.756084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.788846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 480/947] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:54.798286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.803380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83c975-3f31-4bbd-aa1b-f83b6220bda6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.859323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.860037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.860374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.860650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.861441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:54.861753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.863215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.870781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 468/948] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:54.876699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.921702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.952704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.969751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d3498-a906-465c-b3c7-a368d1ff8828 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.032283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.033401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.033869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.034334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.053102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.055915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.056455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1ed8b-c84e-4275-8405-a5550907b42d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-76f12aea-7c54-4c7e-8c2e-1bde6949dea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:15:55.091465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:55.093474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:55.095223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:15:55.095223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:15:55.095223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:15:55.095223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.095223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 481/949] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:55.095814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a61da-7ff4-445d-8ff8-7b7896b47768 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.096315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a61da-7ff4-445d-8ff8-7b7896b47768 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.096770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a61da-7ff4-445d-8ff8-7b7896b47768 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.099506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 482/950] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:55.111369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a61da-7ff4-445d-8ff8-7b7896b47768 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.119833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.120765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.121047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.121444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.122042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:55.122348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.123763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf64d21-1f53-4f73-b881-47cee37f162e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.134994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 469/951] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:55.136951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.137375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.137779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.158634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:55.160650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:55.284155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 470/952] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:55.284155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.310264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a61da-7ff4-445d-8ff8-7b7896b47768 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.339874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:55.357112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:55.385815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.386968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.387556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.388178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.389596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:55.390167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.393163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.434321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 483/953] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:55.436130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.442808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8b88e-1b4b-4bb1-8894-9e2c39f8941f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.506468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.507791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.508535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.510228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.510954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.517164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.548303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 471/954] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:15:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:55.548303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.638385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1ec77-dd04-413b-ad85-504304028a1f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.669631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:55.686456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae67c0-69d6-48b2-a327-3d459dab4d91 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.711476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.712543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.712972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.713413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.714636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.717839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.719908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING keystone.server.flask.common [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 18:15:55.733641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.735488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.736276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.737067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.746550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.sql.core [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:15:55.746550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 18:15:55.746550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [parameters: {'id': '5a3237f1c1084271a4cc512a371736dc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 18:15:55.746550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80147) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:15:55.751476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.754449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a3237f1c1084271a4cc512a371736dc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a3237f1c1084271a4cc512a371736dc. Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:15:55.755094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result.read() Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 18:15:55.756948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application yield resource Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise self.value Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application session.commit() Jun 16 18:15:55.758597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:15:55.760782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:15:55.764315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result.read() Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:15:55.767370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [parameters: {'id': '5a3237f1c1084271a4cc512a371736dc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:55.770610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:55.773977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a3237f1c1084271a4cc512a371736dc. Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 484/955] 10.222.0.30 () {60 vars in 1140 bytes} [Thu Jun 16 18:15:55 2022] POST /identity/v3/projects => generated 262 bytes in 322 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:55.777953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:55.791355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:55.794424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:15:55.799100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 472/956] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:55.800818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.866165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:56.000319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d46a51-fa3e-4183-aae6-e545bd866a91 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:56.007704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff147774-e00e-401c-8451-f263c1287b92 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:56.059268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:56.060765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.061248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.061796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.074520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:56.077184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:56.082381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:56.083544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.084065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.084661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.086188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:56.086747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:56.090482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:15:56.104809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:56.106792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:15:56.108976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:56.114220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 485/957] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:15:56.115377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 473/958] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:56.128610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:56.130149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:56.172163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:56.313629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede60df-a6f7-4fb1-96a6-79f1016ea6fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:56.348481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:56.357056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e8da2-03d2-41db-9e05-7c918b86b166 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:56.368993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03600d-78f8-4b89-97fe-b3b9d8ce62ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:56.370230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03600d-78f8-4b89-97fe-b3b9d8ce62ec None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.370663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03600d-78f8-4b89-97fe-b3b9d8ce62ec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.371128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03600d-78f8-4b89-97fe-b3b9d8ce62ec None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/93c4664744bf4bfbbc5436c5866fcb92/roles/2a17f711a8a648448b0285630ea71d9a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.374781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03600d-78f8-4b89-97fe-b3b9d8ce62ec None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=93c4664744bf4bfbbc5436c5866fcb92, role_id=2a17f711a8a648448b0285630ea71d9a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:56.377144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03600d-78f8-4b89-97fe-b3b9d8ce62ec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:56.406119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 474/959] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:15:56 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/93c4664744bf4bfbbc5436c5866fcb92/roles/2a17f711a8a648448b0285630ea71d9a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:56.450869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:56.451939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.452357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.452679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.454183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:56.454788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:56.457232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350355b-a561-4fcc-8307-b654ab7e71ae None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:56.491512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 486/960] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:15:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:56.854357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefc309-19bd-4f19-8362-330693bae7ce None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.854809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefc309-19bd-4f19-8362-330693bae7ce None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.855171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefc309-19bd-4f19-8362-330693bae7ce None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.857409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 475/961] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:56.864079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8373a9-c8d6-4de6-bfd8-9e1162114fd4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.864753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8373a9-c8d6-4de6-bfd8-9e1162114fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.864753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8373a9-c8d6-4de6-bfd8-9e1162114fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.889843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9d8373a9-c8d6-4de6-bfd8-9e1162114fd4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:56.891415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9d8373a9-c8d6-4de6-bfd8-9e1162114fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:56.992188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 487/962] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:57.028292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefc309-19bd-4f19-8362-330693bae7ce None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:57.189332 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:57.244177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefc309-19bd-4f19-8362-330693bae7ce None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:57.302985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad08f5c-af51-41de-a4b4-868a1ef52916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:57.304025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad08f5c-af51-41de-a4b4-868a1ef52916 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:57.304336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad08f5c-af51-41de-a4b4-868a1ef52916 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:57.304616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad08f5c-af51-41de-a4b4-868a1ef52916 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:57.305947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad08f5c-af51-41de-a4b4-868a1ef52916 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:57.308916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad08f5c-af51-41de-a4b4-868a1ef52916 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:57.335793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 476/963] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:15:57 2022] POST /identity/v3/services => generated 242 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:58.157043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e5c6-8312-46ba-9d4f-ccc82cda605f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:58.158139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e5c6-8312-46ba-9d4f-ccc82cda605f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:58.158748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e5c6-8312-46ba-9d4f-ccc82cda605f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:58.160884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 488/964] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:58.167820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:58.168927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:58.169439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:58.189777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:58.191001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:58.310534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 477/965] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:58.334363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945e5c6-8312-46ba-9d4f-ccc82cda605f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:58.453654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:58.472816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945e5c6-8312-46ba-9d4f-ccc82cda605f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:58.523197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:58.524212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:58.524582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:58.525118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:58.526092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:58.527735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:15:58.543640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:15:58.545639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:15:58.548273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 489/966] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:15:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:15:58.555402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:58.659140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:58.736289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf095066-f1a1-42fe-bec6-fa7de273b85c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:58.800562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:58.801805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:58.802274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:58.802740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:58.803733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:58.804293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:58.806408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:58.813747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 478/967] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:58.821055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:58.971216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493d558-c82d-4435-a323-29eb998661d0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:59.016942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:59.018086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.018572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.019066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.020114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:59.020572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:59.022516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:59.033390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 490/968] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:15:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:59.043408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:59.174419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5d8ca-d1db-4919-b05f-d94a22907df4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:59.213617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:59.227815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:59.231196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.231530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.231988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.233053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] RBAC: Adding query filter params (interface=admin, service_id=cc362852efd24b619ed34e2018519d3e, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:59.233429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:59.236311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b22b42-c650-4230-9393-c94ee799f1cb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:59.244880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 479/969] 10.222.0.30 () {56 vars in 1249 bytes} [Thu Jun 16 18:15:59 2022] GET /identity/v3/endpoints?service_id=cc362852efd24b619ed34e2018519d3e&interface=admin®ion_id=RegionOne => generated 193 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:59.251510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:59.284463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:59.411043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e92b6f-919b-4913-94e8-49fba7c87f78 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:59.460492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f86c7-8507-4334-ac42-d8e7865c81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:59.461703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f86c7-8507-4334-ac42-d8e7865c81f0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.462677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f86c7-8507-4334-ac42-d8e7865c81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.463487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f86c7-8507-4334-ac42-d8e7865c81f0 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.464412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f86c7-8507-4334-ac42-d8e7865c81f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:59.466374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f86c7-8507-4334-ac42-d8e7865c81f0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:59.475214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 491/970] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:15:59 2022] GET /identity/v3/services => generated 1727 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:59.705071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d99b50-de88-46b5-bb2d-ef3b5ce52a33 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.705448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d99b50-de88-46b5-bb2d-ef3b5ce52a33 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.705743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d99b50-de88-46b5-bb2d-ef3b5ce52a33 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.707563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 480/971] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:15:59.713870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.714195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.714500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.731335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3ae43-5089-455b-a084-e2e4e41ae261 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.732221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3ae43-5089-455b-a084-e2e4e41ae261 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.732904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3ae43-5089-455b-a084-e2e4e41ae261 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.736336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 481/972] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:15:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:15:59.736684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:59.738991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:59.741973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:59.742728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:59.743013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:59.771538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:59.773547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:59.846729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 492/973] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:15:59.863164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 482/974] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:15:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:15:59.877817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:59.889625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:59.897805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:00.017538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacb927-b750-4969-80c2-589a5c5e381d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.040038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:00.054202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989f596-2b62-469a-b70f-41a13ff25c26 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.064178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.064899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.065229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.065539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.066410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:00.068026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:00.069787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING keystone.server.flask.common [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 18:16:00.087979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.sql.core [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:16:00.087979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 18:16:00.087979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [parameters: {'user_id': 'ec8bfe8283a8446e95f4382bbbf223d6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 18:16:00.087979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:16:00.107961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.109415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.110170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.110622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 16 18:16:00.111761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:16:00.113302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application yield resource Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise self.value Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application session.commit() Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:16:00.114414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:16:00.115514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:16:00.117443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:16:00.121778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ec8bfe8283a8446e95f4382bbbf223d6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:00.124608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 18:16:00.126678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 483/975] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Jun 16 18:15:59 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:00.128960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:00.146142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:00.157728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:00.158784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:00.159636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:00.162943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 493/976] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Jun 16 18:15:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:00.170448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:00.277757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721419c8-e25f-4050-9f3d-574b82d90723 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.292033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:00.334836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c631c42-852a-485a-8321-986e0eab0668 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.340375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.341572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.342064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.342507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.356728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:00.359458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7a815c52-c2c5-4fec-915f-2f97ccdce353 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:16:00.377502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:00.378775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.380772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 484/977] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:00.381653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a02493-81a5-4944-a995-f45712717088 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.381653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a02493-81a5-4944-a995-f45712717088 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.381653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a02493-81a5-4944-a995-f45712717088 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.388168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 485/978] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:00.388398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a02493-81a5-4944-a995-f45712717088 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:00.390958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.392041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.392699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.393982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.394257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:00.394744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:00.397089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e99f-fde2-4928-b096-95ef08f93c86 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:00.412792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 494/979] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:00.415666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.416145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.416834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.445575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:00.448276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:00.458748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:00.557013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a02493-81a5-4944-a995-f45712717088 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.559476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 495/980] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:00.571221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:00.613034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.614062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.614479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.614911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.616266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:00.616761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:00.619280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:00.666887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 486/981] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:00.667766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:00.680570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:00.760516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71bda-9532-489b-b1bd-04aa3c6cc412 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.811159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.811873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.812185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.812458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.825623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:00.828346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:00.830113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:00.842996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5b94-74a4-4422-a3d9-92bd3e5274d0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:00.852354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:00.853854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:00.854822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:00.854822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:00.854822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:00.854822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:00.854822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 496/982] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:00.864436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:00.897925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:00.898745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:00.899022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:00.899297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:00.901444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:00.902857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:00.918814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:00.919969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:00.920873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 487/983] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:00.926681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.009982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c23b-2e97-44e0-8f94-47cdef11be78 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.030322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.059812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.060522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.060801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.061074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.061856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:01.062112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.063480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.076724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 497/984] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:01.082107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.107527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c827d4-4265-41f6-bc85-cab0c9388ce2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.159059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.160359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.160749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.161093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.162256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:01.162589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.164507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.171712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 488/985] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:16:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:01.178192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.208629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.215040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d40183-a18a-4b89-8264-8bdbf93d2de6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.255041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.256035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.256486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.256866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.265719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.267155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:16:01.283796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:01.285031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:01.285989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 498/986] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:01.293110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.299134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31cac-f75d-41f2-bd9c-4756c7265699 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.332097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.348294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.349518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.349796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.350299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.351280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:01.351870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.354027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.359782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 489/987] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:01.366229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.415375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e849a-fe92-48eb-ad7e-b42b24ab2e4d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.457457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.458174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.458500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.458737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.459599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:01.459913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.462373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.478402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.481613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 499/988] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:01.487230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.497226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3024a66-f398-4e9b-be7a-9d39ce8c6e22 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.546769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.547477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.547759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.548033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.548853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] RBAC: Adding query filter params (interface=public, service_id=6a13eea6926f4f5c9771c97db529c5fa, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:01.549174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.550537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.556811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 490/989] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/endpoints?service_id=6a13eea6926f4f5c9771c97db529c5fa&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:01.562503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.600073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.613221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9076f9-d69b-4c01-9f90-34c5d3e5db47 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.656732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a5cd-bbcf-4cf7-814f-6914affabc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.657504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a5cd-bbcf-4cf7-814f-6914affabc5f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.657789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a5cd-bbcf-4cf7-814f-6914affabc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.658066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a5cd-bbcf-4cf7-814f-6914affabc5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.677235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3824-c52d-4de5-9ac3-dfa9022f4de2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:01.715479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:01.716188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:01.716472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:01.716749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:01.717429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.718787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.725623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 491/990] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:16:01.730854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a5cd-bbcf-4cf7-814f-6914affabc5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:01.732380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a5cd-bbcf-4cf7-814f-6914affabc5f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:01.842898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.859425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 500/991] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:01.860784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:01.884466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:01.987195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54706e1d-569a-4d4b-9efe-3c98fc1387bb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.026227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.027319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.027769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.028232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.040950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.042723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.048924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79466-d9b7-4267-b9ad-2309be6879ce None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.049207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79466-d9b7-4267-b9ad-2309be6879ce None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.049483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79466-d9b7-4267-b9ad-2309be6879ce None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.050852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 501/992] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:02.056809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.057285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.057575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:02.064274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:02.065365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:02.066490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:02.066490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:02.066490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:02.066490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:02.068143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 492/993] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:02.073713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.080248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:02.081460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:02.159984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:02.168959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 502/994] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:02.195465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.211826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0089c-7012-48ca-b8bf-f9b021125710 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.256356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.257081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.257374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.257623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.258356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:02.258686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.260810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.278652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 493/995] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:02.280088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.324480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630231ce-f5ad-4cd2-93cb-ba8991efec54 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.350845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:02.365536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.366462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.366677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.366964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.367803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.369584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:02.383738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:02.385502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:02.389099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 503/996] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:02.391727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.402251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7911-5930-47c7-a866-a80610056c4d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.433207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:02.441648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.442456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.442735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.443157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.450672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.452214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:16:02.467921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:02.469123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:02.470315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 494/997] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:02.475032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.518518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3387a-b2dd-4c44-bab1-eb391d444c91 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.558997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.559675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.559953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.560341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.560986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:02.561297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.562720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.574150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 504/998] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:02.575646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.601218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea1a7a-98a0-44fd-a365-4ac24f14db51 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.632185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:02.642691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.643484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.643867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.644273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.645165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:02.645570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.647094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.661416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 495/999] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:02.669906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.712679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edf06b-3864-4b35-b438-def41631d392 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.726995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:02.764510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.765870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.766499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.767124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.768682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:02.769369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.772240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.780666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 505/1000] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:02.787060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:02.810733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec150-6564-40c6-bfa5-29fe521f8290 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.867390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:02.868163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:02.869146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:02.869429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:02.870345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] RBAC: Adding query filter params (role.id=2a17f711a8a648448b0285630ea71d9a, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=93c4664744bf4bfbbc5436c5866fcb92) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:02.870584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:02.872039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013973c7-ee24-44e7-8d2b-7b3447e19ab8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:02.884795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 496/1001] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:02 2022] GET /identity/v3/role_assignments?user.id=93c4664744bf4bfbbc5436c5866fcb92&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=2a17f711a8a648448b0285630ea71d9a => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:02.943470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deefcb-be87-4ac4-aa26-1b8ac73581ee None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:02.972294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:03.015082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c71d2f-1c69-437f-a96e-57c86c5ad6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:03.016494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c71d2f-1c69-437f-a96e-57c86c5ad6eb None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:03.017136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c71d2f-1c69-437f-a96e-57c86c5ad6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:03.017782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c71d2f-1c69-437f-a96e-57c86c5ad6eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:03.019426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c71d2f-1c69-437f-a96e-57c86c5ad6eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:03.022737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c71d2f-1c69-437f-a96e-57c86c5ad6eb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:03.056482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 506/1002] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:16:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:04.091316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5783-599b-4e92-a7b0-1c1eb5cd2442 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:04.091859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5783-599b-4e92-a7b0-1c1eb5cd2442 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:04.091859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5783-599b-4e92-a7b0-1c1eb5cd2442 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:04.093847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 497/1003] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:04.100087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:04.100561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:04.100639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:04.121999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:04.123301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:04.240349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 507/1004] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:04.262886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5783-599b-4e92-a7b0-1c1eb5cd2442 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:04.419613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5783-599b-4e92-a7b0-1c1eb5cd2442 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:04.436653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:04.460669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:04.461577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:04.462379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:04.462657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:04.463341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:04.464738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:04.477335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:04.479285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:04.480696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 498/1005] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:16:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:04.485240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:04.597443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:04.601173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5900f-66e6-4d5a-bf81-382af8c13b86 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:04.666839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:04.667984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:04.668529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:04.669023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:04.670405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:04.670902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:04.673485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:04.687678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 508/1006] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:16:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:04.689122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:04.805975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241163b-1f07-4737-9bab-b795e6beab6f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:04.842571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:04.843255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:04.843537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:04.843808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:04.844553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] RBAC: Adding query filter params (type=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:04.844859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:04.846218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20ebce-c07d-4e40-bd0e-fe5864fa58df None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:04.851931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 499/1007] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Jun 16 18:16:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:04.859121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:04.974723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea224d6-4867-4a15-936a-87ae8e2d6b3b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:05.017727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b22c76-96c1-4981-b593-1d3f9eb8bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:05.018396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22c76-96c1-4981-b593-1d3f9eb8bc2d None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.018675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22c76-96c1-4981-b593-1d3f9eb8bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.018949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22c76-96c1-4981-b593-1d3f9eb8bc2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.019692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b22c76-96c1-4981-b593-1d3f9eb8bc2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:05.021090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b22c76-96c1-4981-b593-1d3f9eb8bc2d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:05.043540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 509/1008] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:16:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:05.271745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2095-edb0-42ce-adb5-7abaf29016b0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.272664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2095-edb0-42ce-adb5-7abaf29016b0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.272950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2095-edb0-42ce-adb5-7abaf29016b0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.275266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 500/1009] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:05.281009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.281526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.281871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.301697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:05.303042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:05.323577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:05.376913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56911c50-ab85-4362-bc8e-ead901da0e9d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.377240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56911c50-ab85-4362-bc8e-ead901da0e9d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.377474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56911c50-ab85-4362-bc8e-ead901da0e9d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.379174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 501/1010] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:05.385974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.386299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.386526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.406944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:05.408573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:05.417329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 510/1011] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:05 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:05.441443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:05.459508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:05.512103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 502/1012] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:05 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:05.537114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:05.576898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b3f4e-e11b-4c11-8ba2-661b9554aaba None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:05.619103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:05.619933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.620255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.620532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.621217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:05.622634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:05.635931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:05.637527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:05.638550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:05.642169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 511/1013] 10.222.0.30 () {56 vars in 1105 bytes} [Thu Jun 16 18:16:05 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:05.643091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:05.658709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:05.661808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545ab20-c332-4e30-9ab7-b82ed6cdee0c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:05.708500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:05.709322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.709605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.709881 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.711097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:05.713049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:05.714915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 503/1014] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:05.720456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:05.765667 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:05.773789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4620a6f-fa89-4687-b7ac-6be7690108d0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:05.815199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:05.815804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.816117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.816399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.817117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] RBAC: Adding query filter params (name=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:05.817428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:05.818811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:05.825962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 512/1015] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:05.833320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:05.844280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42ea9-217d-45c9-a07f-fbc929259fe6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:05.885654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b5b2d-07a0-46a9-b683-f566c7c117c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:05.886743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b5b2d-07a0-46a9-b683-f566c7c117c9 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.887224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b5b2d-07a0-46a9-b683-f566c7c117c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.887740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b5b2d-07a0-46a9-b683-f566c7c117c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.888748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b5b2d-07a0-46a9-b683-f566c7c117c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:05.890388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b5b2d-07a0-46a9-b683-f566c7c117c9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:05.916821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:05.934691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 504/1016] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:16:05 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:05.949085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7221b6-83c3-4fc8-be3b-5e155d30fb38 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:05.989396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:05.990112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:05.990399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:05.990680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:05.991447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] RBAC: Adding query filter params (type=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:05.991739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:05.993932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a7e15-f48d-45be-b020-72f29593f197 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:06.008352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 513/1017] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:07.406521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cfcd8-3673-4b84-8a7d-f71f84688640 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:07.407325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cfcd8-3673-4b84-8a7d-f71f84688640 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:07.407325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cfcd8-3673-4b84-8a7d-f71f84688640 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:07.408866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 505/1018] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:07.413868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:07.414249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:07.414406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:07.431391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:07.432590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:07.510778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 514/1019] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:07 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:07.532484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cfcd8-3673-4b84-8a7d-f71f84688640 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:07.628289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:07.641798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cfcd8-3673-4b84-8a7d-f71f84688640 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:07.679637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:07.680406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:07.681624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:07.681965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:07.693243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:07.701712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:07.702775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:07.703647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:07.707896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 506/1020] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Jun 16 18:16:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:07.709432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:07.782096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:07.833508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2373c7-841c-43d5-85ac-abd2db98912b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:07.880157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:07.881167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:07.881623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:07.882057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:07.883208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:07.883651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:07.885787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:07.895077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 515/1021] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:16:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:07.901138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.007821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9893c-abe1-4fd4-8a6d-79a6d532cda6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.044404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.045223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.045511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.045788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.056749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.058216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4b93f555-681d-479d-a8c4-4e5837f822d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:08.077194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:08.078139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:08.084120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:08.084120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:08.084120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:08.084120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:08.084120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 507/1022] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:08.084418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.201596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325caef-f038-4510-839f-eaeb9b47714d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.201891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325caef-f038-4510-839f-eaeb9b47714d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.202171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325caef-f038-4510-839f-eaeb9b47714d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.203841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 508/1023] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:08.209541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a68b2-18a0-43b6-9a34-25e17aa7dd18 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.209864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a68b2-18a0-43b6-9a34-25e17aa7dd18 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.210101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a68b2-18a0-43b6-9a34-25e17aa7dd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.216622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.231283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cc864-600c-4c3a-882c-d0037e762909 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.231854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-060a68b2-18a0-43b6-9a34-25e17aa7dd18 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:08.233114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-060a68b2-18a0-43b6-9a34-25e17aa7dd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:08.265338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.272166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.272794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.273083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.273372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.275337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:08.275337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.276710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed060-2c1b-4b40-92b2-177cf780a16f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.289081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 516/1024] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:08.291031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d42a4-1d50-462d-9a40-18f0d8c31963 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.291544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d42a4-1d50-462d-9a40-18f0d8c31963 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.291719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d42a4-1d50-462d-9a40-18f0d8c31963 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.294819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 517/1025] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:08.295614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d42a4-1d50-462d-9a40-18f0d8c31963 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.309346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 509/1026] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:08 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:08.310649 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.310938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.311235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.339231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:08.340752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:08.417448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d42a4-1d50-462d-9a40-18f0d8c31963 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.419200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 510/1027] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:08 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:08.420132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.457171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.457971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.458259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.458533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.466056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.467493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:16:08.482091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:08.484315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 518/1028] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:16:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:08.486496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.489968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.540920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8063ea5-cedd-4f22-8110-efb13af34afd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.545701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.576678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.577348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.577636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.577911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.578640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.580006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.593631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 511/1029] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:16:08 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:08.594515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.596824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec802a78-ec58-4047-8bf0-17e8435ef710 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.647223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.648085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.648365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.648693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.650794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.652880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.654823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 519/1030] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:08.661018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.729083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6667eda-bdc5-4e70-90a2-0d9ea708ec6a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.754106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.768103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.768798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.769082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.769356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.770247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:08.770565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.771998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.786991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc510-aacb-473c-87d0-ae72f48bf29b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.792789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 512/1031] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:16:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:08.798736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.804005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.828463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.829147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.829429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.829703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.830424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.831954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.868810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 520/1032] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:16:08 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:08.932765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8787-caa1-4379-b1e7-965168763572 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.974042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.974717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.975024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.975302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:09.033991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:09.058990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:09.060403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:09.192834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 513/1033] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:09.199590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:09.312447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a83e94-acae-49fa-89dd-48ff4a569f69 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:09.354156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:09.354985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:09.355265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:09.355538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:09.367984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:09.369800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:09.389829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:09.390830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:09.391764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:09.391764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:09.391764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:09.391764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:09.400323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 521/1034] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:09.400540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:09.497741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:09.530547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2e7a6-4bdb-48a4-bc6c-781b1384c06f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:09.570051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:09.570655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:09.570943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:09.571785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:09.572599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:09.572910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:09.574342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:09.588722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 514/1035] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:09.594203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:09.622989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:09.711746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379efc73-af41-48a4-8ec3-8ae0a224324d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:09.752363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:09.753231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:09.753522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:09.753799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:09.762388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:09.763850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:16:09.781266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:09.782488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:09.783435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:09.788023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 522/1036] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:09.788725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:09.901212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a82d9-6e38-4034-82ca-ec5faba6ab92 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:09.941015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:09.941708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:09.941989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:09.942278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:09.943061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:09.943472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:09.945109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6237d3-3209-4d04-a92a-6a6cb283aa07 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:09.956581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:09.964555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 515/1037] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:09.970609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:10.084428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:10.120091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b27c-72a6-4353-88b7-deb0a048e347 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:10.191117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:10.191783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:10.192145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:10.192544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:10.193697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] RBAC: Adding query filter params (role.id=2a17f711a8a648448b0285630ea71d9a, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=793065092530407983e5da2446d2e939) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:10.194081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:10.195619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400be33-66e8-45b5-85ac-a9dab073774b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:10.208168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 523/1038] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:09 2022] GET /identity/v3/role_assignments?user.id=793065092530407983e5da2446d2e939&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=2a17f711a8a648448b0285630ea71d9a => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:10.835750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663e82-818a-4305-9cd5-8ef62c665679 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:10.836320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663e82-818a-4305-9cd5-8ef62c665679 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:10.836588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663e82-818a-4305-9cd5-8ef62c665679 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:10.838771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 516/1039] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:10.844530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:10.845020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:10.845098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:10.870724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:10.872732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:10.961626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 524/1040] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:10.985392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c663e82-818a-4305-9cd5-8ef62c665679 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.119997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c663e82-818a-4305-9cd5-8ef62c665679 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.157755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.158558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.158866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.159125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] PATH_INFO: `/v3/services/image` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.159784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:11.161200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9760c916-f986-40bb-91f4-46c1f9720a5a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:11.173739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:11.174899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:11.175777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:11.177421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 517/1041] 10.222.0.30 () {56 vars in 1105 bytes} [Thu Jun 16 18:16:10 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:11.181873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.261827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737728b-a0d8-4263-b4c8-66fe9ccd7d65 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.262311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737728b-a0d8-4263-b4c8-66fe9ccd7d65 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.262596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737728b-a0d8-4263-b4c8-66fe9ccd7d65 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.264217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 518/1042] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:11.269401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.269691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.269968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.288265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:11.292599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:11.293812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:11.320564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3c9b-115d-4dd4-8b95-3b5af351f141 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.365699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:11.369782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.371372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.372224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.372518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.373602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] RBAC: Adding query filter params (name=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:11.373917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:11.375368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:11.378430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 519/1043] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:11.386733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 525/1044] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:11.394092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.404558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.539745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc9573-9df7-444c-b5bb-1cbbc064f249 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.540601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f462d26-653a-495a-a194-6529f08eff9c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.592276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.593338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.593777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.593950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.594368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.594756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.595183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.595605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.596454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:11.596832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:11.598707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:11.605077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 520/1045] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:11.605937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:11.615525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:11.615767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:11.625260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:11.626238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:11.627182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 526/1046] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:11.634904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.692946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:11.750587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd660e-2286-41b8-b481-e8de09b820ce None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.772733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d3d26-ff0e-4ba7-aced-f8aec11aeb96 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.789483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.790773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.791056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.791328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.792146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] RBAC: Adding query filter params (interface=public, service_id=a90b7d36d3f441d38b5a9540b0d5289f, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:11.792459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:11.793825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:11.799241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 521/1047] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/endpoints?service_id=a90b7d36d3f441d38b5a9540b0d5289f&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:11.807415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.818501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.819774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.820253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.820729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.821689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:11.822171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:11.823727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:11.833534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 527/1048] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:11.839941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:11.912735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:11.937388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484304e1-56f4-4109-aa6c-b4b6cfbabdd8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.963697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140561d1-bf69-4199-83fa-f6e943192dff None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:11.968994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:11.985325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:11.986455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:11.986745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:11.987024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:11.987720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:11.989759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:12.008081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 522/1049] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/services => generated 1957 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:16:12.008295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61641141-53fc-44b9-90d9-a8e5c216c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:12.008295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641141-53fc-44b9-90d9-a8e5c216c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.008295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641141-53fc-44b9-90d9-a8e5c216c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.008295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641141-53fc-44b9-90d9-a8e5c216c1d8 None admin] PATH_INFO: `/v3/users/b9486e16803a4c3b971b1a2827369cd8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.022650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641141-53fc-44b9-90d9-a8e5c216c1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=b9486e16803a4c3b971b1a2827369cd8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:12.024206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641141-53fc-44b9-90d9-a8e5c216c1d8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:12.026859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 528/1050] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:11 2022] GET /identity/v3/users/b9486e16803a4c3b971b1a2827369cd8 => generated 285 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:12.033349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:12.187901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7df45-e453-4a35-b2c0-afbed1b5d172 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:12.239385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:12.240563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.241064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.241549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.345294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:12.371178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:12.373876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:12.518240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3c0be-3fe1-40be-aedd-5c7992be28d3 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.518637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3c0be-3fe1-40be-aedd-5c7992be28d3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.518820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3c0be-3fe1-40be-aedd-5c7992be28d3 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.520371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 529/1051] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:12.523538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 523/1052] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:16:12.527153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.527439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.527717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.529787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:12.545421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:12.546634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:12.630423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 530/1053] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:12.638364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:12.655124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:12.665954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b221-c6a7-4a7d-b006-20715a21df56 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:12.708956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:12.709829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.710113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.710389 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.711588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] RBAC: Authorizing `identity:get_project(project_id=f34d230a4a3141ff80d126a5836fad0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:12.714327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:12.716107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 524/1054] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:12 2022] GET /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:12.722319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:12.741728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:12.815486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2270a-47a5-448b-a410-a635d7a65e51 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:12.852710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:12.853685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.853962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.854232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.856812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d0855-12d6-4b60-9f47-4634ccd6fced None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:12.865776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:12.874843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:12.875957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:12.877338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 531/1055] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Jun 16 18:16:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:12.882389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:12.903121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:12.903946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:12.904270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:12.904550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:12.905397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=f34d230a4a3141ff80d126a5836fad0d, user.id=b9486e16803a4c3b971b1a2827369cd8) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:12.905714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:12.907504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:12.916728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 525/1056] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:12 2022] GET /identity/v3/role_assignments?user.id=b9486e16803a4c3b971b1a2827369cd8&scope.project.id=f34d230a4a3141ff80d126a5836fad0d&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:13.024479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd695e-23ec-4b3f-9f00-72cbcd8d4248 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:13.045997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:13.075406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:13.075998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:13.076426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:13.076701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:13.077459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:13.077761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:13.079136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:13.089623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 532/1057] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:16:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:13.097023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:13.128496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:13.231892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad209a-e56c-42dc-97c8-9e9821e2a119 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:13.269985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:13.271759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:13.272495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:13.273167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:13.285866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:13.287970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:13.306274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:13.307505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:13.308477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:13.308477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:13.308477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:13.308477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:13.309156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 526/1058] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:13.314183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:13.436796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8346353-8e89-4b79-94e4-ac313588ca88 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:13.479395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:13.480083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:13.480358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:13.480628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:13.481355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:13.481659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:13.483016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:13.496318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 533/1059] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:13.502464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:13.579698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:13.613249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd98e10-bc65-4287-af4a-b53b2bb08738 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:13.646977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:13.647955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:13.648299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:13.648579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:13.658231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:13.659679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:16:13.675206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:13.676858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:16:13.678241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:13.680437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 527/1060] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:16:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:13.682604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:13.717106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:13.813408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce13ff-6383-4013-b1d1-04ecb3370e27 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:13.854550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:13.855282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:13.855559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:13.855829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:13.856738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:13.857044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:13.858603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:13.881313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 534/1061] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:16:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:13.886725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.009765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bb24-8831-4c65-a556-558ef65b3617 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.043794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:14.051515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.052416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.052704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.052985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.067295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.068800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:14.090989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:14.093184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:14.095352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:14.095352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:14.095352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:14.095352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:14.100114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 528/1062] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:14.100796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.273246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:14.286195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385ea9f-82c6-4066-96b4-f47a8c708116 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.329639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.330628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.331074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.331502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.332439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:14.332917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.334780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959578f-91d6-408b-9266-66bf57927dea None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:14.349107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 535/1063] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:14.356757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.388565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda335d-199a-4474-acd3-85ac0e9dd764 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.389030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda335d-199a-4474-acd3-85ac0e9dd764 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.389447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda335d-199a-4474-acd3-85ac0e9dd764 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.392169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 536/1064] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:14.397823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.398181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.398380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.418022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:14.419266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:14.497702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb11ac4-6529-4dc5-9994-7e7ed9a1671c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.515792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 537/1065] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:14.541168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.544253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.545095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.545382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.545662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.553920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.555421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:16:14.575024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:14.577124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:14.579679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 529/1066] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:16:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:14.582146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.645424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:14.653012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:14.678543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3ecc3-b735-4304-8abc-9ee6a889aa27 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.718547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90446096-45e0-4879-ad02-054c40529923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.719222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fca46a-e552-4de6-9b59-ab9e4fc437d7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.719407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90446096-45e0-4879-ad02-054c40529923 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.719683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90446096-45e0-4879-ad02-054c40529923 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.719961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90446096-45e0-4879-ad02-054c40529923 None admin] PATH_INFO: `/v3/services/image` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.720681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90446096-45e0-4879-ad02-054c40529923 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.722063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90446096-45e0-4879-ad02-054c40529923 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-90446096-45e0-4879-ad02-054c40529923 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:14.735031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:14.736181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:14.737027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 538/1067] 10.222.0.30 () {56 vars in 1105 bytes} [Thu Jun 16 18:16:14 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:14.742926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90446096-45e0-4879-ad02-054c40529923 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.764250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.765021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.765308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.765583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.766369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:14.766681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.768141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:14.784120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 530/1068] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:14.789788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.868541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90446096-45e0-4879-ad02-054c40529923 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.911408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.912108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.912407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.912672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.913392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] RBAC: Adding query filter params (name=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:14.913702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.915204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:14.921016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 539/1069] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:14.922941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a6641-bb08-49fe-b44d-c036fbeb1450 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:14.925093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:14.928134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:14.944248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:14.983993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832f035-e794-4636-8534-962d3db29676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:14.984934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f035-e794-4636-8534-962d3db29676 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:14.985224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f035-e794-4636-8534-962d3db29676 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:14.985504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f035-e794-4636-8534-962d3db29676 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/793065092530407983e5da2446d2e939/roles/2a17f711a8a648448b0285630ea71d9a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:14.987920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f035-e794-4636-8534-962d3db29676 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=793065092530407983e5da2446d2e939, role_id=2a17f711a8a648448b0285630ea71d9a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:14.989759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f035-e794-4636-8534-962d3db29676 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:15.019912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 531/1070] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:16:14 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/793065092530407983e5da2446d2e939/roles/2a17f711a8a648448b0285630ea71d9a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:16:15.093425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ffe9b-22a2-4142-89c9-d89ee092da46 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:15.148343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030696-1d23-4caf-9db2-492b8631de06 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.149267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030696-1d23-4caf-9db2-492b8631de06 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.150656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030696-1d23-4caf-9db2-492b8631de06 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.152397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 532/1071] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:15.158636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.159035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.159229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.161095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:15.161782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.162062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.162337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.163063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] RBAC: Adding query filter params (type=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:15.163369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:15.164804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:15.171820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 540/1072] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:15.179639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:15.181075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:15.181706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:15.268363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 533/1073] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:15.274790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:15.296467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:15.311858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528c2bc-31b9-4da3-a632-f8bf560fc030 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:15.357512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:15.358193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.358520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.358802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.359565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:15.361388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:15.401039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 541/1074] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:16:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:15.449698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:15.465794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b5236-ae41-406e-a797-401aa8337c3b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:15.532669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:15.533502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.533782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.534053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.548791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:15.557046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:15.558293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:15.559550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:15.566703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 534/1075] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:15.568623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:15.707049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9013a-7447-4077-8a42-f85a85e6bb94 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:15.739587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:15.751440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:15.752165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.752446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.752718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.753484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:15.753794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:15.755167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:15.765882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 542/1076] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:15.771636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:15.893100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f1dd5-daa0-4af0-9156-24b3dcee7a74 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:15.935758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:15.936162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:15.937097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:15.937097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] PATH_INFO: `/v3/users/b9486e16803a4c3b971b1a2827369cd8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:15.938175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] RBAC: Authorizing `identity:get_user(user_id=b9486e16803a4c3b971b1a2827369cd8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:15.939530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:15.941475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 535/1077] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:15 2022] GET /identity/v3/users/b9486e16803a4c3b971b1a2827369cd8 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:15.949628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:16.108451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34373fd9-4842-46f8-943b-c1c9a01aad75 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:16.161549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c7d1-3beb-4a2c-8ea1-d9ab31736d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:16.162979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c7d1-3beb-4a2c-8ea1-d9ab31736d61 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:16.163442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c7d1-3beb-4a2c-8ea1-d9ab31736d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:16.163913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c7d1-3beb-4a2c-8ea1-d9ab31736d61 None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:16.166711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12c7d1-3beb-4a2c-8ea1-d9ab31736d61 None admin] RBAC: Authorizing `identity:get_project(project_id=f34d230a4a3141ff80d126a5836fad0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:16.169366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12c7d1-3beb-4a2c-8ea1-d9ab31736d61 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:16.172701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 543/1078] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:15 2022] GET /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d => generated 311 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:16.182601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:16.209990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:16.302641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29e62a-86aa-4835-90d7-42ab8a42dba3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:16.339861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e4c4f-90b7-47a5-85b9-62a9065d1163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:16.340807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e4c4f-90b7-47a5-85b9-62a9065d1163 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:16.341081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e4c4f-90b7-47a5-85b9-62a9065d1163 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:16.341355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e4c4f-90b7-47a5-85b9-62a9065d1163 None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d/users/b9486e16803a4c3b971b1a2827369cd8/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:16.343475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e4c4f-90b7-47a5-85b9-62a9065d1163 None admin] RBAC: Authorizing `identity:create_grant(project_id=f34d230a4a3141ff80d126a5836fad0d, user_id=b9486e16803a4c3b971b1a2827369cd8, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:16.344988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e4c4f-90b7-47a5-85b9-62a9065d1163 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:16.366446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 536/1079] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:16:16 2022] PUT /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d/users/b9486e16803a4c3b971b1a2827369cd8/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:17.426292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee4ed2-a8c2-4ef6-b640-8b1358dd5894 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:17.426916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee4ed2-a8c2-4ef6-b640-8b1358dd5894 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:17.426916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee4ed2-a8c2-4ef6-b640-8b1358dd5894 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:17.428508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 544/1080] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:17.434689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea831d5-bc4f-44bc-9270-9f63a75e6a4a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:17.435266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea831d5-bc4f-44bc-9270-9f63a75e6a4a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:17.435266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea831d5-bc4f-44bc-9270-9f63a75e6a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:17.453363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-aea831d5-bc4f-44bc-9270-9f63a75e6a4a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:17.454568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aea831d5-bc4f-44bc-9270-9f63a75e6a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:17.547221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 537/1081] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:17.571030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee4ed2-a8c2-4ef6-b640-8b1358dd5894 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:17.655625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:17.682024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc3844-2047-4dff-b02c-a5c74d1d9e70 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:17.682316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc3844-2047-4dff-b02c-a5c74d1d9e70 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:17.682507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc3844-2047-4dff-b02c-a5c74d1d9e70 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:17.684099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 538/1082] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:17.689222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:17.689518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:17.689803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:17.702179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:17.709310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:17.711544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:17.725909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee4ed2-a8c2-4ef6-b640-8b1358dd5894 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:17.794827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:17.796222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:17.796712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:17.797180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:17.804873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 539/1083] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:17.815533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:17.829029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:17.833243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:17.834208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:16:17.835079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:17.840385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 545/1084] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Jun 16 18:16:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:17.845202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:17.953828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848da504-4cc0-4a17-98e3-55eff3b47438 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:17.964211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab150b-d7eb-47ac-b1f3-c82e1845a7b5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:17.983249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:17.993706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:17.994691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:17.994805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:17.995047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:17.999158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.003323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.004014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.004329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.004610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.005384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:18.005692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.007099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.017049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 546/1085] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:16:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:18.022683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.081234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.082740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.170473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e7f49-1183-42f0-8ae4-e4410abf60ec None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:18.221340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.222042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.222318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.222607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.235601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.237205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.248138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 540/1086] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:18.253914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:18.256703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:18.257590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:18.258494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:18.258494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:18.258494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:18.258494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:18.258494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 547/1087] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:18.264355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.297151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.303476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.373431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611de4-2828-4724-9a58-0800e118c8e2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:18.383139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252fec2-8cc0-40de-a8dd-598dd6ba9f01 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:18.416183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.417550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.417550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.417885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.423558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.424256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.424537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.427679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.427679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:18.427679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.427679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.435375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.437019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.441784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 548/1088] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:18.447523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:18.456968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:18.458102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:18.459738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:18.459738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:18.459738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:18.459738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:18.464460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 541/1089] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:18.466184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.589108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.601547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.605765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033e08d-5ffc-43d7-a91b-49fbfc2f3eac None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:18.617668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2043bff-3980-458e-b51e-bac3ee62ff22 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:18.656260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.657360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.657630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.657898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.661025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.661733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.662017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.662294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.663028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:18.663340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.664886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4748e-49ec-4c89-b27b-ff5136bc4fa1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.667152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.668642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.679592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 542/1090] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:18.681625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7374f9-bf53-4a1c-9fec-63aeeab487fb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.682065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7374f9-bf53-4a1c-9fec-63aeeab487fb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.682496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7374f9-bf53-4a1c-9fec-63aeeab487fb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.688100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 543/1091] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:16:18.689267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:18.690980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:16:18.692012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:18.693735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.693735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.693735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.695633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 549/1092] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:18.696667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.712855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:18.714257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:18.803034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 544/1093] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:18.827366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.842043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1e1ec-7caf-4ceb-943b-100163b69baf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:18.893971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069975e-cd34-424d-8812-affc2059603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:18.895462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069975e-cd34-424d-8812-affc2059603f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:18.896224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069975e-cd34-424d-8812-affc2059603f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:18.896973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069975e-cd34-424d-8812-affc2059603f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:18.898658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069975e-cd34-424d-8812-affc2059603f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:18.899246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069975e-cd34-424d-8812-affc2059603f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:18.902617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069975e-cd34-424d-8812-affc2059603f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:18.943857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 550/1094] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:18.945991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069975e-cd34-424d-8812-affc2059603f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:18.946728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.967541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:18.988687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cfdc1-316a-481b-a9aa-d708cb533430 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.043868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.044722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.045006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.045277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.057908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:19.065656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:19.067591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:19.069329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 545/1095] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:19.073650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.091143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069975e-cd34-424d-8812-affc2059603f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.144293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.145349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.145624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.145973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.195370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf469d-6d35-4043-9c71-940b99b95719 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.226447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.228514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.234230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.235335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.235691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.236260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.237898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:19.238437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.241078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.241367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.249013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.256393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 546/1096] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:19.259912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.360970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 551/1097] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:19.367267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.381403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08381a-ce84-4f93-ab87-fe4a6fb5e7d9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.424491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.425380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.425652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.425919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] PATH_INFO: `/v3/users/b9486e16803a4c3b971b1a2827369cd8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.427121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] RBAC: Authorizing `identity:get_user(user_id=b9486e16803a4c3b971b1a2827369cd8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.428757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.431336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 547/1098] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/users/b9486e16803a4c3b971b1a2827369cd8 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:19.437211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.474628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568f601-f284-4922-97a9-8872b98436e5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.486953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.507271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.520258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.521153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.521506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.521901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.534393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.535881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.545371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a129f-9a2e-4886-aae0-ea261aa6e881 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:19.554979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:19.556680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:19.557599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:16:19.557599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:19.557599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:16:19.557599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:19.557599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 552/1099] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:19.562265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.584871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.585529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.585814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.586086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.663724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.665872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.668467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaee6d-6f07-4ccf-bdf6-072e4d1a1c0e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.710746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.711422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.711702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.712538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.713430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:19.713743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.715133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.727422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 553/1100] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:19.736560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.751297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.771265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.796350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 548/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:19.802193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.862605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71232fa-0bd0-4808-b266-b5511968c0de None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.897657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.898463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.898744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.899023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.906743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.908470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.908599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4459d-681e-43af-a773-8305279d3d20 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:16:19.923152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:19.924945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:19.926766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 554/1102] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:19.929800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.948991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.949853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.950139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.950417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.951630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] RBAC: Authorizing `identity:get_project(project_id=f34d230a4a3141ff80d126a5836fad0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.953214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.955255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 549/1103] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:19.960860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.996112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:20.015155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:20.038203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983adbc-320c-4481-a5e6-087040b6e28d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:20.063365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fbafd-2582-473f-9cfe-a6183097700a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:20.074485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:20.075157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:20.075439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:20.075711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:20.076533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:20.076845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:20.078273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:20.090537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 555/1104] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:20.096932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:20.104728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:20.105758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:20.106045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:20.106319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:20.107165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=f34d230a4a3141ff80d126a5836fad0d, user.id=b9486e16803a4c3b971b1a2827369cd8) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:20.107485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:20.108940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd28836-15e2-48b4-988b-0971bc9b5b31 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:20.115320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 550/1105] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/role_assignments?user.id=b9486e16803a4c3b971b1a2827369cd8&scope.project.id=f34d230a4a3141ff80d126a5836fad0d&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:20.217657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de7e6c-a29f-44f9-8b0b-b8ed03e980fd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:20.257109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:20.257772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:20.258049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:20.258320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:20.259202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] RBAC: Adding query filter params (role.id=2a17f711a8a648448b0285630ea71d9a, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=793065092530407983e5da2446d2e939) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:20.259619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:20.261097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a946-9656-4b88-98b3-90fcbfe01372 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:20.267944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 556/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:20 2022] GET /identity/v3/role_assignments?user.id=793065092530407983e5da2446d2e939&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=2a17f711a8a648448b0285630ea71d9a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:22.313052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b66d5b-ef26-4376-8a18-25c3be535951 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.313591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b66d5b-ef26-4376-8a18-25c3be535951 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.313591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b66d5b-ef26-4376-8a18-25c3be535951 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.314980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 551/1107] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:22.320114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.320856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.320950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.332467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:22.338180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:22.339386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:22.407828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 557/1108] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:22.433689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b66d5b-ef26-4376-8a18-25c3be535951 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:22.535752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:22.541182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b66d5b-ef26-4376-8a18-25c3be535951 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:22.574050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e5689-d74d-4f69-a6d8-b6407ff4c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:22.574768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e5689-d74d-4f69-a6d8-b6407ff4c117 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.575050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e5689-d74d-4f69-a6d8-b6407ff4c117 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.575321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e5689-d74d-4f69-a6d8-b6407ff4c117 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.576516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e5689-d74d-4f69-a6d8-b6407ff4c117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:22.577975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e5689-d74d-4f69-a6d8-b6407ff4c117 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:22.579585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 552/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:22.585461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:22.683647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e384c-ac6b-46ca-bf70-883d224079e1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:22.717929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d3be0-3d3a-4049-b4a8-2739059baae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:22.718606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d3be0-3d3a-4049-b4a8-2739059baae0 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.718892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d3be0-3d3a-4049-b4a8-2739059baae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.719174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d3be0-3d3a-4049-b4a8-2739059baae0 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.719907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d3be0-3d3a-4049-b4a8-2739059baae0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:22.721555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d3be0-3d3a-4049-b4a8-2739059baae0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:22.729893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:22.743190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 558/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:16:22 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:24.936748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4276717-f7da-4fda-99cd-673cd4d2eead None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:24.937317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4276717-f7da-4fda-99cd-673cd4d2eead None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:24.937317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4276717-f7da-4fda-99cd-673cd4d2eead None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:24.938760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 553/1111] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:24.944126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:24.944782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:24.944782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:24.962068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:24.963298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:25.034096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 559/1112] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:25.058645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4276717-f7da-4fda-99cd-673cd4d2eead None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:25.157418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4276717-f7da-4fda-99cd-673cd4d2eead None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:25.191609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:25.192426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:25.192707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:25.192981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] PATH_INFO: `/v3/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:25.194450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] RBAC: Authorizing `identity:get_role(role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:25.195803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:25.197724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 554/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:25 2022] GET /identity/v3/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:25.203537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:25.307233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169150-f549-4023-a44a-7c25f4d3dee3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:25.332182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:25.345160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:25.346042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:25.346322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:25.346596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] PATH_INFO: `/v3/users/35aacdf286204f6ab1943a72f2836fd7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:25.362002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=35aacdf286204f6ab1943a72f2836fd7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:25.363450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:25.365181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 560/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:25 2022] GET /identity/v3/users/35aacdf286204f6ab1943a72f2836fd7 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:25.373217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:25.436820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:25.506553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e586a-5e09-48ca-8e0e-e49ea7931cd2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:25.545561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:25.547027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:25.547027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:25.547027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:25.620328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:25.621782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:25.655528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:25.723158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 555/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:25.730740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:25.842083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330807bf-ccd7-4472-94b9-5abfb5266ee4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:25.879730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642d41c-0500-45a7-8974-ac2ca7fad881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:25.880628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d41c-0500-45a7-8974-ac2ca7fad881 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:25.880905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d41c-0500-45a7-8974-ac2ca7fad881 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:25.881180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d41c-0500-45a7-8974-ac2ca7fad881 None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:25.882372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642d41c-0500-45a7-8974-ac2ca7fad881 None admin] RBAC: Authorizing `identity:get_project(project_id=f34d230a4a3141ff80d126a5836fad0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:25.884474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642d41c-0500-45a7-8974-ac2ca7fad881 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:25.886708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 561/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:25 2022] GET /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:25.892923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:25.995954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341d2f4-3444-411d-9b43-56078dd4d5dc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:26.030901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:26.031561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:26.031842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:26.032407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:26.033242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] RBAC: Adding query filter params (role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=f34d230a4a3141ff80d126a5836fad0d, user.id=35aacdf286204f6ab1943a72f2836fd7) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:26.033552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:26.034980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0eb64d-f194-4b21-abbc-77a5759c0245 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:26.038078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:26.041688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 556/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:25 2022] GET /identity/v3/role_assignments?user.id=35aacdf286204f6ab1943a72f2836fd7&scope.project.id=f34d230a4a3141ff80d126a5836fad0d&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:28.189875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69269cb-f9c7-4983-97f8-1382d0b86cf2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.190477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69269cb-f9c7-4983-97f8-1382d0b86cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.190477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69269cb-f9c7-4983-97f8-1382d0b86cf2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.191791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 562/1118] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:28.197219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.197708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.197808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.214555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:28.215770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:28.284466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 557/1119] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:28.308503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69269cb-f9c7-4983-97f8-1382d0b86cf2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:28.370890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:28.427920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69269cb-f9c7-4983-97f8-1382d0b86cf2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:28.464106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:28.464885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.465166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.465430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] PATH_INFO: `/v3/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.466902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] RBAC: Authorizing `identity:get_role(role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:28.468299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:28.469670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 563/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:28 2022] GET /identity/v3/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:28.475904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:28.580385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419df745-7e6c-43eb-a022-648d0cdadd72 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:28.610195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:28.616530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:28.617416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.617696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.618504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] PATH_INFO: `/v3/users/35aacdf286204f6ab1943a72f2836fd7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.620338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] RBAC: Authorizing `identity:get_user(user_id=35aacdf286204f6ab1943a72f2836fd7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:28.621769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:28.623407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 558/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:28 2022] GET /identity/v3/users/35aacdf286204f6ab1943a72f2836fd7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:28.629361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:28.727597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f0430-271f-490c-b44a-759c55b188fb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:28.744626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:28.763816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f221a8-73bf-4fab-8139-27a61a126039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:28.764703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f221a8-73bf-4fab-8139-27a61a126039 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.764992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f221a8-73bf-4fab-8139-27a61a126039 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.765267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f221a8-73bf-4fab-8139-27a61a126039 None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.766452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f221a8-73bf-4fab-8139-27a61a126039 None admin] RBAC: Authorizing `identity:get_project(project_id=f34d230a4a3141ff80d126a5836fad0d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:28.767884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f221a8-73bf-4fab-8139-27a61a126039 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:28.769610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 564/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:28 2022] GET /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:28.775712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:28.873845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36926f92-3c55-4478-9e5b-c57e172675b8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:28.906803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6615d76-f065-44b4-9f4d-da717e36253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:28.907939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6615d76-f065-44b4-9f4d-da717e36253b None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.908239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6615d76-f065-44b4-9f4d-da717e36253b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.908512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6615d76-f065-44b4-9f4d-da717e36253b None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d/users/35aacdf286204f6ab1943a72f2836fd7/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.910669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6615d76-f065-44b4-9f4d-da717e36253b None admin] RBAC: Authorizing `identity:create_grant(project_id=f34d230a4a3141ff80d126a5836fad0d, user_id=35aacdf286204f6ab1943a72f2836fd7, role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:28.912284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6615d76-f065-44b4-9f4d-da717e36253b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:28.926276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 559/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:16:28 2022] PUT /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d/users/35aacdf286204f6ab1943a72f2836fd7/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:31.132753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455314-50fd-4705-9f6b-5f8f3a428af5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:31.133343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455314-50fd-4705-9f6b-5f8f3a428af5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:31.133343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455314-50fd-4705-9f6b-5f8f3a428af5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:31.134678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 565/1124] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:31.140268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:31.140875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:31.140875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:31.157613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:31.158821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:31.213381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:31.230606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 560/1125] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:31.255558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455314-50fd-4705-9f6b-5f8f3a428af5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:31.374787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455314-50fd-4705-9f6b-5f8f3a428af5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:31.411821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:31.412631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:31.412909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:31.413180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] PATH_INFO: `/v3/roles/4c6bbdc3b8af4bc1b5c1a4986617096f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:31.414673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] RBAC: Authorizing `identity:get_role(role_id=4c6bbdc3b8af4bc1b5c1a4986617096f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:31.416462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:31.417851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 566/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:31 2022] GET /identity/v3/roles/4c6bbdc3b8af4bc1b5c1a4986617096f => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:31.423702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:31.527658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e0cc9-dcd9-4f51-bce2-078199799121 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:31.562381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:31.563260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:31.563536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:31.563808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] PATH_INFO: `/v3/users/35aacdf286204f6ab1943a72f2836fd7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:31.565052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] RBAC: Authorizing `identity:get_user(user_id=35aacdf286204f6ab1943a72f2836fd7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:31.566492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:31.568154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 561/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:31 2022] GET /identity/v3/users/35aacdf286204f6ab1943a72f2836fd7 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:31.575110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:31.629181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:31.677518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269068ef-5ffc-4644-99de-0a47e6f5f94f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:31.714236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:31.714894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:31.715168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:31.715442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:31.785706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:31.787141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:31.842490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:31.900587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 567/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:16:31.906766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:31.966314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:32.014776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be046092-1d17-44e7-a448-9562cc9d4c02 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:32.051505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8257d-cdfd-4308-abe5-08e0cc2a0404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:32.052503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8257d-cdfd-4308-abe5-08e0cc2a0404 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:32.052833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8257d-cdfd-4308-abe5-08e0cc2a0404 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:32.053170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8257d-cdfd-4308-abe5-08e0cc2a0404 None admin] PATH_INFO: `/v3/projects/f34d230a4a3141ff80d126a5836fad0d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:32.054654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8257d-cdfd-4308-abe5-08e0cc2a0404 None admin] RBAC: Authorizing `identity:get_project(project_id=f34d230a4a3141ff80d126a5836fad0d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:32.056229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8257d-cdfd-4308-abe5-08e0cc2a0404 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:32.058016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 562/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:31 2022] GET /identity/v3/projects/f34d230a4a3141ff80d126a5836fad0d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:32.064415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:32.170071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1946c-54ae-4a4d-9e95-e483bc4ddac4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:32.205182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:32.205840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:32.206123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:32.206401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:32.207236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] RBAC: Adding query filter params (role.id=4c6bbdc3b8af4bc1b5c1a4986617096f, scope.project.id=f34d230a4a3141ff80d126a5836fad0d, user.id=35aacdf286204f6ab1943a72f2836fd7) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:32.207551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:32.209010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d752809-0158-486d-91b0-f5a548eea1a8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:32.214994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 568/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:32 2022] GET /identity/v3/role_assignments?user.id=35aacdf286204f6ab1943a72f2836fd7&scope.project.id=f34d230a4a3141ff80d126a5836fad0d&role.id=4c6bbdc3b8af4bc1b5c1a4986617096f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:34.401980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c7eda-0d5c-4a82-baf6-eb7336883e9e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.402549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c7eda-0d5c-4a82-baf6-eb7336883e9e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.402549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c7eda-0d5c-4a82-baf6-eb7336883e9e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.403884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 563/1131] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:34.409501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.409950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.410049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.426491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:34.427694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:34.439128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:34.496688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 569/1132] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:34.520146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c7eda-0d5c-4a82-baf6-eb7336883e9e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.624290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c7eda-0d5c-4a82-baf6-eb7336883e9e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.646712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:34.659091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749760-618f-4d7b-8d42-d423c6869e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.659821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749760-618f-4d7b-8d42-d423c6869e7c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.660137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749760-618f-4d7b-8d42-d423c6869e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.660419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749760-618f-4d7b-8d42-d423c6869e7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.661600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749760-618f-4d7b-8d42-d423c6869e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.663036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749760-618f-4d7b-8d42-d423c6869e7c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.664728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 564/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:34.671561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.769503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34594feb-8850-4259-9c23-6bdd254e4cf9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.805629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe81-0fb8-4f9a-82bb-bc0f801b41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.806386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe81-0fb8-4f9a-82bb-bc0f801b41c7 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.806714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe81-0fb8-4f9a-82bb-bc0f801b41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.806993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe81-0fb8-4f9a-82bb-bc0f801b41c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.807736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbe81-0fb8-4f9a-82bb-bc0f801b41c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.809261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbe81-0fb8-4f9a-82bb-bc0f801b41c7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.831920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:34.841959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 570/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:16:34 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:37.081968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fad49f-4dce-4573-8fd5-892d64f3dc91 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:37.082520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fad49f-4dce-4573-8fd5-892d64f3dc91 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:37.082520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fad49f-4dce-4573-8fd5-892d64f3dc91 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:37.083910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 565/1135] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:37.089883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:37.090320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:37.090447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:37.107178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:37.108412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:37.175769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 571/1136] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:37.201219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fad49f-4dce-4573-8fd5-892d64f3dc91 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:37.304508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fad49f-4dce-4573-8fd5-892d64f3dc91 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:37.344729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88555ffd-9847-4bfa-8fe9-718fa42f82f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:37.345545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88555ffd-9847-4bfa-8fe9-718fa42f82f6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:37.345827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88555ffd-9847-4bfa-8fe9-718fa42f82f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:37.346098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88555ffd-9847-4bfa-8fe9-718fa42f82f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:37.347327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88555ffd-9847-4bfa-8fe9-718fa42f82f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:37.348822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88555ffd-9847-4bfa-8fe9-718fa42f82f6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:37.350524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 566/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:37.356914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:37.463344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86e37-ae91-42ae-b3c0-c8879ed732b3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:37.488991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:37.498047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aad8e-3b50-4fd8-9561-77fb3b2cc882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:37.498741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aad8e-3b50-4fd8-9561-77fb3b2cc882 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:37.499031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aad8e-3b50-4fd8-9561-77fb3b2cc882 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:37.499307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aad8e-3b50-4fd8-9561-77fb3b2cc882 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:37.500079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aad8e-3b50-4fd8-9561-77fb3b2cc882 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:37.501629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aad8e-3b50-4fd8-9561-77fb3b2cc882 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:37.524523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 572/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:16:37 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:39.819478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f482fc-1c15-4eeb-8cc7-560a87fa43b1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:39.819983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f482fc-1c15-4eeb-8cc7-560a87fa43b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:39.819983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f482fc-1c15-4eeb-8cc7-560a87fa43b1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:39.821579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 567/1139] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:39.827146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:39.827604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:39.827755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:39.846498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:39.847692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:39.915760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 573/1140] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:39.939359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f482fc-1c15-4eeb-8cc7-560a87fa43b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.002863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:40.048016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f482fc-1c15-4eeb-8cc7-560a87fa43b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.082652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.083233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.083561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.083786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.096061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:40.103739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:40.104979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:40.106114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 568/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:40.111460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.209941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24597eb8-3472-4fd0-9bfa-23481415d55b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.224499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:40.245587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.246261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.246554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.246830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.247581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:40.247889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.249298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.257773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 574/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:40.264678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.364243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d8b4-f07e-45a1-91d2-2fa9a3aa50ff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.385665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:40.407876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.408805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.409085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.409359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] PATH_INFO: `/v3/users/4a57915feea14e61867ecd929fec0c4a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.428335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=4a57915feea14e61867ecd929fec0c4a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.430279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.433610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 569/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:40 2022] GET /identity/v3/users/4a57915feea14e61867ecd929fec0c4a => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:40.439591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.546367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259a32-3eb7-43ed-8cb0-8bebf87a4447 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.581790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.582440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.582725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.583005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.631757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:40.659525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.661026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.766489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 575/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:16:40.772998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.875566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11e06c-3869-4021-ac40-d088adabc8c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.911014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec14f5-9201-4a81-b7f3-12a8382ad709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.911803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec14f5-9201-4a81-b7f3-12a8382ad709 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.912112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec14f5-9201-4a81-b7f3-12a8382ad709 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.912392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec14f5-9201-4a81-b7f3-12a8382ad709 None admin] PATH_INFO: `/v3/projects/773c0b8022d444298f9db88d8ce2f62a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.913569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec14f5-9201-4a81-b7f3-12a8382ad709 None admin] RBAC: Authorizing `identity:get_project(project_id=773c0b8022d444298f9db88d8ce2f62a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.915535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec14f5-9201-4a81-b7f3-12a8382ad709 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.917268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 570/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:40 2022] GET /identity/v3/projects/773c0b8022d444298f9db88d8ce2f62a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:40.923411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:41.000784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:41.024265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257e90-e3ca-4af7-8889-74c84997ed5e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:41.060535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.061186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.061469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.061745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.062570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=773c0b8022d444298f9db88d8ce2f62a, user.id=4a57915feea14e61867ecd929fec0c4a) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:41.062881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.064325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff9a60-9e16-4334-8172-347018023398 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:41.069791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 576/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:40 2022] GET /identity/v3/role_assignments?user.id=4a57915feea14e61867ecd929fec0c4a&scope.project.id=773c0b8022d444298f9db88d8ce2f62a&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:43.205826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c13656-ea79-42f8-97fe-65db22e493f5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:43.206346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c13656-ea79-42f8-97fe-65db22e493f5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:43.206463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c13656-ea79-42f8-97fe-65db22e493f5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:43.207746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 571/1147] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:43.213253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:43.213779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:43.213779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:43.231164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:43.232399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:43.301707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 577/1148] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:43.325187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c13656-ea79-42f8-97fe-65db22e493f5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:43.376879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:43.437781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c13656-ea79-42f8-97fe-65db22e493f5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:43.472616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:43.473334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:43.473621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:43.473899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:43.484888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:43.492671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:43.494334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:43.495223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 572/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:43.500790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:43.572416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:43.603492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed573-0b34-49a3-8d3d-6c71711f89dd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:43.639468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:43.640173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:43.640456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:43.640730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:43.641485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:43.641791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:43.643162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:43.652462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 578/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:43.658813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:43.767795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9806-e678-4aca-a508-601b13f6bc07 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:43.775335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:43.806299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:43.807188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:43.807475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:43.807750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] PATH_INFO: `/v3/users/4a57915feea14e61867ecd929fec0c4a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:43.809027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] RBAC: Authorizing `identity:get_user(user_id=4a57915feea14e61867ecd929fec0c4a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:43.810472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:43.812157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 573/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:43 2022] GET /identity/v3/users/4a57915feea14e61867ecd929fec0c4a => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:43.818062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:43.925254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca863d-aa39-4278-8f31-b896cfbb9cf9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:43.954295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:43.959801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfd533-4464-4008-91af-194c6b669a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:43.960699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfd533-4464-4008-91af-194c6b669a17 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:43.960982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfd533-4464-4008-91af-194c6b669a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:43.961255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfd533-4464-4008-91af-194c6b669a17 None admin] PATH_INFO: `/v3/projects/773c0b8022d444298f9db88d8ce2f62a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:43.962480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfd533-4464-4008-91af-194c6b669a17 None admin] RBAC: Authorizing `identity:get_project(project_id=773c0b8022d444298f9db88d8ce2f62a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:43.963919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfd533-4464-4008-91af-194c6b669a17 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:43.965659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 579/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:43 2022] GET /identity/v3/projects/773c0b8022d444298f9db88d8ce2f62a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:43.971792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:44.072712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbb7a4-a83d-479c-8a55-23ee614df7d8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:44.108370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46ff73-f4d3-4313-b317-4a401ef3cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:44.109275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46ff73-f4d3-4313-b317-4a401ef3cc7b None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:44.109560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46ff73-f4d3-4313-b317-4a401ef3cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:44.109839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46ff73-f4d3-4313-b317-4a401ef3cc7b None admin] PATH_INFO: `/v3/projects/773c0b8022d444298f9db88d8ce2f62a/users/4a57915feea14e61867ecd929fec0c4a/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:44.111804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46ff73-f4d3-4313-b317-4a401ef3cc7b None admin] RBAC: Authorizing `identity:create_grant(project_id=773c0b8022d444298f9db88d8ce2f62a, user_id=4a57915feea14e61867ecd929fec0c4a, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:44.113284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46ff73-f4d3-4313-b317-4a401ef3cc7b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:44.128606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 574/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:16:43 2022] PUT /identity/v3/projects/773c0b8022d444298f9db88d8ce2f62a/users/4a57915feea14e61867ecd929fec0c4a/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:46.294046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c5a26-be20-417b-a966-8155fd7ff33d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:46.294591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c5a26-be20-417b-a966-8155fd7ff33d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:46.294708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c5a26-be20-417b-a966-8155fd7ff33d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:46.296237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 580/1154] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:46.302511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:46.302961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:46.303418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:46.328482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:46.330085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:46.366822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:46.412132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 575/1155] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:46.436850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c5a26-be20-417b-a966-8155fd7ff33d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:46.581890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c5a26-be20-417b-a966-8155fd7ff33d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:46.626482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:46.627319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:46.627598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:46.627873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:46.642153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:16:46.650345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:46.652147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:16:46.653589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 581/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:16:46.659053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:46.786255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270faa-3f53-425b-8e7d-b7f526d08326 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:46.827473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:46.828516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:46.828938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:46.829347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:46.830498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:46.830967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:46.833188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:46.846648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 576/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:16:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:46.855127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:46.928129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:46.972496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cb523-bd83-415f-a2e8-8996a00a9f8a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:47.010980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:47.011863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:47.012171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:47.012446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] PATH_INFO: `/v3/users/4a57915feea14e61867ecd929fec0c4a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:47.013731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] RBAC: Authorizing `identity:get_user(user_id=4a57915feea14e61867ecd929fec0c4a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:47.015188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:47.016898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 582/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:16:46 2022] GET /identity/v3/users/4a57915feea14e61867ecd929fec0c4a => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:47.022910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:47.051652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:47.127657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e11e8c-e973-4e2d-9f9d-a9aabb7899bc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:47.163022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:47.163683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:47.163965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:47.164275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:47.235953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:47.237444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:47.266786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:47.344288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 577/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:16:47.350523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:47.463562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613f85-873f-44a8-9bc2-f2142ab598d9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:47.491162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:47.500379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef5d7d-c09e-481f-85a2-e15085329ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:47.501256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef5d7d-c09e-481f-85a2-e15085329ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:47.501540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef5d7d-c09e-481f-85a2-e15085329ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:47.501814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef5d7d-c09e-481f-85a2-e15085329ce0 None admin] PATH_INFO: `/v3/projects/773c0b8022d444298f9db88d8ce2f62a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:47.503042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef5d7d-c09e-481f-85a2-e15085329ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=773c0b8022d444298f9db88d8ce2f62a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:47.504530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef5d7d-c09e-481f-85a2-e15085329ce0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:47.506260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 583/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:16:47 2022] GET /identity/v3/projects/773c0b8022d444298f9db88d8ce2f62a => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:47.513324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:47.619717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcfaea-2435-4ba9-8b64-14b61a45ce61 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:47.647905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:47.657160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:47.658083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:47.658364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:47.658640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:47.659476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=773c0b8022d444298f9db88d8ce2f62a, user.id=4a57915feea14e61867ecd929fec0c4a) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:47.659792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:47.661273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf25a4-0907-4fb3-8021-227e2d9946b5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:47.668689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 578/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:16:47 2022] GET /identity/v3/role_assignments?user.id=4a57915feea14e61867ecd929fec0c4a&scope.project.id=773c0b8022d444298f9db88d8ce2f62a&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:49.916156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa6df-b721-422e-adbf-036f7b1770db None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:49.916736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa6df-b721-422e-adbf-036f7b1770db None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:49.916736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa6df-b721-422e-adbf-036f7b1770db None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:49.918203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 584/1162] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:49.924156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:49.924674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:49.924794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:49.942961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:49.944208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:50.019210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 579/1163] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:50.043763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa6df-b721-422e-adbf-036f7b1770db None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.170246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa6df-b721-422e-adbf-036f7b1770db None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.207837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.208531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.208813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.209085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.278488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:50.286556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.288023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.402919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 585/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:16:50.409477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.532104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151829-e100-4ba9-8c0a-b8d42c143d29 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.554687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:50.570126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.571517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.571940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.572325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.579376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.580847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-cad8b347-7f7f-46dc-a018-e12917412603 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:16:50.596115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:16:50.597487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:16:50.598383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 580/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:16:50.603997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.705102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:50.711531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd773c8-6fb7-40f1-8953-613f31bfaf57 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.749452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.750122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.750402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.750674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.751407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:50.751713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.753138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93b84c-9434-40ee-8946-3c13d693ecff None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.761665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 586/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:52.984557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38030-3033-4210-8165-ba990100ebb5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.985152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38030-3033-4210-8165-ba990100ebb5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.985152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe38030-3033-4210-8165-ba990100ebb5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.986693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 581/1167] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:16:52.994015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762be64-bed3-4a83-95a7-35446bee3a8a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.994584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762be64-bed3-4a83-95a7-35446bee3a8a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.994584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762be64-bed3-4a83-95a7-35446bee3a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:53.013797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a762be64-bed3-4a83-95a7-35446bee3a8a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:53.015050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a762be64-bed3-4a83-95a7-35446bee3a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:53.106199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 587/1168] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:53.129972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe38030-3033-4210-8165-ba990100ebb5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:53.302444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe38030-3033-4210-8165-ba990100ebb5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:53.355628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf98a8-ac34-4d85-9130-17d39a0689e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:53.356469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf98a8-ac34-4d85-9130-17d39a0689e8 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:53.356838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf98a8-ac34-4d85-9130-17d39a0689e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:53.357211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf98a8-ac34-4d85-9130-17d39a0689e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:53.358046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf98a8-ac34-4d85-9130-17d39a0689e8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:53.359505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf98a8-ac34-4d85-9130-17d39a0689e8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:53.376144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 582/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Thu Jun 16 18:16:53 2022] POST /identity/v3/domains => generated 267 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:55.647099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e37d4-d04a-4d62-bddc-8bbc68e79606 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:55.647704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e37d4-d04a-4d62-bddc-8bbc68e79606 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:55.647704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e37d4-d04a-4d62-bddc-8bbc68e79606 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:55.650002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 588/1170] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:55.656200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:55.656711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:55.656846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:55.675100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:55.676350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:55.714976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:55.758146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 583/1171] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:16:55.783499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e37d4-d04a-4d62-bddc-8bbc68e79606 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:55.895450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:55.908709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e37d4-d04a-4d62-bddc-8bbc68e79606 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:55.954217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c4538-6aa8-48a2-99be-4b1069e1416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:55.955121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c4538-6aa8-48a2-99be-4b1069e1416e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:55.955403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c4538-6aa8-48a2-99be-4b1069e1416e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:55.955677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c4538-6aa8-48a2-99be-4b1069e1416e None admin] PATH_INFO: `/v3/domains/c3ccd03f96f240d18832f4ca1dc417d6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:55.968415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c4538-6aa8-48a2-99be-4b1069e1416e None admin] RBAC: Authorizing `identity:get_domain(domain_id=c3ccd03f96f240d18832f4ca1dc417d6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:55.969909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c4538-6aa8-48a2-99be-4b1069e1416e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:55.971732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 589/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Thu Jun 16 18:16:55 2022] GET /identity/v3/domains/c3ccd03f96f240d18832f4ca1dc417d6 => generated 267 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:55.980092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:56.125347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe08d38-50c7-4a62-8418-c995fa840ac1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:56.174244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15a425-a484-44f8-aef9-de0406c5d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:56.174924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a425-a484-44f8-aef9-de0406c5d022 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:56.175209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a425-a484-44f8-aef9-de0406c5d022 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:56.175486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a425-a484-44f8-aef9-de0406c5d022 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:56.176237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a425-a484-44f8-aef9-de0406c5d022 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:56.177684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a425-a484-44f8-aef9-de0406c5d022 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:56.208570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 584/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:16:55 2022] POST /identity/v3/projects => generated 361 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:58.425764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cfff9-377f-4963-a3ab-ea4f8433f22f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:58.426423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cfff9-377f-4963-a3ab-ea4f8433f22f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:58.426423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cfff9-377f-4963-a3ab-ea4f8433f22f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:58.427663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 590/1174] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:16:58.433398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:58.433845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:58.433950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:58.451489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:58.452732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:58.469845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:58.526763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 585/1175] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:16:58.551177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cfff9-377f-4963-a3ab-ea4f8433f22f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:58.656748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cfff9-377f-4963-a3ab-ea4f8433f22f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:58.690589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f317f2-adbd-4bcb-87f5-0c5c29170c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:58.691363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f317f2-adbd-4bcb-87f5-0c5c29170c28 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:58.691644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f317f2-adbd-4bcb-87f5-0c5c29170c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:58.691919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f317f2-adbd-4bcb-87f5-0c5c29170c28 None admin] PATH_INFO: `/v3/domains/c3ccd03f96f240d18832f4ca1dc417d6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:58.693130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f317f2-adbd-4bcb-87f5-0c5c29170c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c3ccd03f96f240d18832f4ca1dc417d6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:58.694564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f317f2-adbd-4bcb-87f5-0c5c29170c28 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:58.696871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 591/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Thu Jun 16 18:16:58 2022] GET /identity/v3/domains/c3ccd03f96f240d18832f4ca1dc417d6 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:58.702710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:58.801797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf545b0-b98f-4d8f-8bb3-512eb6fd5e29 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:58.836356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3117a-13d7-4ec7-af2a-75522008d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:58.837125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3117a-13d7-4ec7-af2a-75522008d902 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:58.837409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3117a-13d7-4ec7-af2a-75522008d902 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:58.837686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3117a-13d7-4ec7-af2a-75522008d902 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:58.838406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3117a-13d7-4ec7-af2a-75522008d902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:58.839845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3117a-13d7-4ec7-af2a-75522008d902 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:58.860307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 586/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:16:58 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:01.057832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b05f5-02cb-480b-bfb0-1a0b417b6c67 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.058410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b05f5-02cb-480b-bfb0-1a0b417b6c67 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.058410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b05f5-02cb-480b-bfb0-1a0b417b6c67 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.059745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 592/1178] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:17:01.065254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.065733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.065835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.083802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:01.085053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:01.088169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:01.166719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 587/1179] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:17:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:17:01.191069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b05f5-02cb-480b-bfb0-1a0b417b6c67 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.216523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:01.312171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b05f5-02cb-480b-bfb0-1a0b417b6c67 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.351791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.353261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.353540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.353808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.365454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:17:01.373590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:17:01.374813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:17:01.375924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 593/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:17:01.381396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.486638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941a9cc-2ea2-4bfa-9f62-56d6e53e9b60 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.527344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78f932-012b-4d54-8790-105035653994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.528023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78f932-012b-4d54-8790-105035653994 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.528341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78f932-012b-4d54-8790-105035653994 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.528620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78f932-012b-4d54-8790-105035653994 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.529374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78f932-012b-4d54-8790-105035653994 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:01.529718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78f932-012b-4d54-8790-105035653994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.531092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78f932-012b-4d54-8790-105035653994 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.536845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:01.541393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 588/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:01.546942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.627913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:01.654119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88872-1c6c-4b69-be04-e319ec978f31 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.688743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.689619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.689901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.690176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] PATH_INFO: `/v3/users/f07934d12ac44bfa997fb41033dcb707` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.703538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] RBAC: Authorizing `identity:get_user(user_id=f07934d12ac44bfa997fb41033dcb707)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.705024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.706761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 594/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/users/f07934d12ac44bfa997fb41033dcb707 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:01.717645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78f932-012b-4d54-8790-105035653994 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.824371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78f932-012b-4d54-8790-105035653994 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.863772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.864478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.864765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.865040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.931160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:01.933616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.935387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:02.042008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 589/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:17:02.048020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:02.147595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbea2b-07a0-4ffd-830e-3735f19e775d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:02.182107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4aec1-fe29-48de-8c71-e3e2472b7555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:02.182901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4aec1-fe29-48de-8c71-e3e2472b7555 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:02.183193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4aec1-fe29-48de-8c71-e3e2472b7555 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:02.183471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4aec1-fe29-48de-8c71-e3e2472b7555 None admin] PATH_INFO: `/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:02.184734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4aec1-fe29-48de-8c71-e3e2472b7555 None admin] RBAC: Authorizing `identity:get_project(project_id=6c4cfc674a864dff902e3f53a8f3b24b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:02.186169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4aec1-fe29-48de-8c71-e3e2472b7555 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:02.187883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 595/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:17:02 2022] GET /identity/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:02.194018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:02.291606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:02.294842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025f210-91b7-4fc7-94d2-115973d538db None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:02.330816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:02.331503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:02.331789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:02.332092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:02.332950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=6c4cfc674a864dff902e3f53a8f3b24b, user.id=f07934d12ac44bfa997fb41033dcb707) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:02.333267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:02.334730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6c2d-e0cd-4882-a173-f15af9db8919 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:02.341307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 590/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:17:02 2022] GET /identity/v3/role_assignments?user.id=f07934d12ac44bfa997fb41033dcb707&scope.project.id=6c4cfc674a864dff902e3f53a8f3b24b&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:04.529632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500ee19-1b80-41a5-a982-9005d09b9786 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:04.530165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500ee19-1b80-41a5-a982-9005d09b9786 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:04.530165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500ee19-1b80-41a5-a982-9005d09b9786 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:04.531546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 596/1186] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:17:04.536966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:04.537432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:04.537549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:04.555636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:04.556899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:04.625137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 591/1187] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:17:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:17:04.649507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500ee19-1b80-41a5-a982-9005d09b9786 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:04.654893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:04.759773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500ee19-1b80-41a5-a982-9005d09b9786 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:04.796076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:04.796895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:04.797180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:04.797456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:04.807844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:17:04.815566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:17:04.816754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:17:04.817651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 597/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:17:04.823785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:04.921102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:04.928177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d9cf8-7d21-4cef-badf-7be5021d927f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:04.964075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:04.964755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:04.965040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:04.965316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:04.966066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:04.966374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:04.967743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:04.976028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 592/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:17:04.982544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:05.050206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:05.091908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59260d3e-bf51-4720-b362-d195147c81e3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:05.134073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:05.134707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:05.134957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:05.135266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] PATH_INFO: `/v3/users/f07934d12ac44bfa997fb41033dcb707` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:05.136565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] RBAC: Authorizing `identity:get_user(user_id=f07934d12ac44bfa997fb41033dcb707)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:05.138003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:05.139648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 598/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:17:04 2022] GET /identity/v3/users/f07934d12ac44bfa997fb41033dcb707 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:05.145443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:05.260361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e7a0-d24a-4d50-9513-4daf4ad84365 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:05.301189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad271a7-9c06-402b-abac-eeaf2559e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:05.301986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad271a7-9c06-402b-abac-eeaf2559e03f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:05.302264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad271a7-9c06-402b-abac-eeaf2559e03f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:05.302534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad271a7-9c06-402b-abac-eeaf2559e03f None admin] PATH_INFO: `/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:05.303907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad271a7-9c06-402b-abac-eeaf2559e03f None admin] RBAC: Authorizing `identity:get_project(project_id=6c4cfc674a864dff902e3f53a8f3b24b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:05.305457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad271a7-9c06-402b-abac-eeaf2559e03f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:05.307929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 593/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:17:05 2022] GET /identity/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:05.314466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:05.429367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2d021-c558-4039-9f53-f9be4982bc62 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:05.460486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:05.466891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8ed07-8a1e-42e9-9ade-8b7e382d5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:05.467793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8ed07-8a1e-42e9-9ade-8b7e382d5de1 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:05.468108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8ed07-8a1e-42e9-9ade-8b7e382d5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:05.468389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8ed07-8a1e-42e9-9ade-8b7e382d5de1 None admin] PATH_INFO: `/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b/users/f07934d12ac44bfa997fb41033dcb707/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:05.470387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8ed07-8a1e-42e9-9ade-8b7e382d5de1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c4cfc674a864dff902e3f53a8f3b24b, user_id=f07934d12ac44bfa997fb41033dcb707, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:05.471823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8ed07-8a1e-42e9-9ade-8b7e382d5de1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:05.487408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 599/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:17:05 2022] PUT /identity/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b/users/f07934d12ac44bfa997fb41033dcb707/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:07.686218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b2b1-1aba-428a-915a-0d726d1a7e6e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:07.686785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b2b1-1aba-428a-915a-0d726d1a7e6e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:07.687141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b2b1-1aba-428a-915a-0d726d1a7e6e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:07.689643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 594/1193] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:17:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:17:07.695331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:07.695917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:07.695917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:07.715221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:07.716485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:07.787012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 600/1194] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:17:07.811811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7b2b1-1aba-428a-915a-0d726d1a7e6e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:07.840232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:07.924807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7b2b1-1aba-428a-915a-0d726d1a7e6e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:07.963914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:07.964754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:07.965036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:07.965315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:07.978624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:17:07.986296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:17:07.987657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:17:07.990657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 595/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:17:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:17:07.995501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:08.102883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b206180-4ca2-47b9-9968-e1cc61341330 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:08.129511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:08.143341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:08.144071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:08.144361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:08.144639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:08.145417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:08.145738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:08.147128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:08.157270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 601/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:17:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:17:08.163511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:08.256385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:08.278240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b496fcb-82c8-41ef-afe3-cacbf4dfa658 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:08.314986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:08.315870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:08.316184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:08.316463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] PATH_INFO: `/v3/users/f07934d12ac44bfa997fb41033dcb707` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:08.317723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] RBAC: Authorizing `identity:get_user(user_id=f07934d12ac44bfa997fb41033dcb707)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:08.319154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:08.320855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 596/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Jun 16 18:17:08 2022] GET /identity/v3/users/f07934d12ac44bfa997fb41033dcb707 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:08.326910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:08.436776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5178d4e-def1-48a9-af90-b4b1e87a2e8d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:08.470802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:08.471452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:08.471737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:08.472014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:08.525909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:08.542347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:08.543803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:08.644999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 602/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:17:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:17:08.651752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:08.754518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63b15e-c08a-43ec-be66-92e6ea2745f5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:08.790491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e6cc6-b2b7-4835-b52a-13b27723b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:08.791368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e6cc6-b2b7-4835-b52a-13b27723b805 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:08.791650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e6cc6-b2b7-4835-b52a-13b27723b805 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:08.791922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e6cc6-b2b7-4835-b52a-13b27723b805 None admin] PATH_INFO: `/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:08.793401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e6cc6-b2b7-4835-b52a-13b27723b805 None admin] RBAC: Authorizing `identity:get_project(project_id=6c4cfc674a864dff902e3f53a8f3b24b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:08.794859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e6cc6-b2b7-4835-b52a-13b27723b805 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:08.796633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 597/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Jun 16 18:17:08 2022] GET /identity/v3/projects/6c4cfc674a864dff902e3f53a8f3b24b => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:08.802790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:08.889643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:08.906221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5be01-2be2-4d46-8265-525b8e193f96 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:08.941332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:08.941995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:08.942278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:08.942554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:08.943388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=6c4cfc674a864dff902e3f53a8f3b24b, user.id=f07934d12ac44bfa997fb41033dcb707) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:08.943702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:08.945175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed724c7-4213-4e66-93eb-fe22ea6599ab None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:08.952996 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 603/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:17:08 2022] GET /identity/v3/role_assignments?user.id=f07934d12ac44bfa997fb41033dcb707&scope.project.id=6c4cfc674a864dff902e3f53a8f3b24b&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:18.785900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:18.950083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=26 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:10.479983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf92e4b-1d66-47fa-a869-2b82b9edc602 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.480710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf92e4b-1d66-47fa-a869-2b82b9edc602 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.480710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf92e4b-1d66-47fa-a869-2b82b9edc602 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.482268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 598/1201] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:18:10.488363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.489121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.489121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.509240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:10.510499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:10.630093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 604/1202] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:10.657699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf92e4b-1d66-47fa-a869-2b82b9edc602 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:10.818884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf92e4b-1d66-47fa-a869-2b82b9edc602 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:10.868843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:10.869772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.870062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.870371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.892627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:10.894234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:10.933092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:10.935532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:10.938002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:10.938002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:10.938002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:10.938002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:10.941468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 599/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:10.949130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.125688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3926e-9bd5-44a3-a7cf-a001dd98f6e2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.159834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:11.168269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.169319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.169931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.170584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.171479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:11.171823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.173819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526cfce-7860-495b-978b-6ca213aee5d4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.193985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 605/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:11.195271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.274645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:11.364578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5a441-8eaa-4608-910e-3b47d9561954 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.425524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d27f62-81cf-4f06-8e92-f89823eb8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.426588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d27f62-81cf-4f06-8e92-f89823eb8036 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.427146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d27f62-81cf-4f06-8e92-f89823eb8036 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.427560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d27f62-81cf-4f06-8e92-f89823eb8036 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.428894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d27f62-81cf-4f06-8e92-f89823eb8036 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.432415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d27f62-81cf-4f06-8e92-f89823eb8036 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.484087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 600/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Jun 16 18:18:11 2022] POST /identity/v3/users => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:14.475458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457fa5-2949-45c6-afa0-5c7e9ffe6d89 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:14.476469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457fa5-2949-45c6-afa0-5c7e9ffe6d89 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:14.476469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457fa5-2949-45c6-afa0-5c7e9ffe6d89 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:14.478239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 606/1206] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:14.485578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:14.486342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:14.486666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:14.512395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:14.513589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:14.629834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 601/1207] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:14.659362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2457fa5-2949-45c6-afa0-5c7e9ffe6d89 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:14.790510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2457fa5-2949-45c6-afa0-5c7e9ffe6d89 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:14.837483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:14.840417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:14.840851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:14.841301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:14.862093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:18:14.876403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:14.878318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:18:14.880769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:14.884634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 607/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:18:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:18:14.889648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:14.941065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:15.047912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92335767-277d-411f-a335-5e9f90c6c06b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:15.097910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:15.098750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:15.099083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:15.099364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:15.100493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:15.100910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:15.102603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:15.114531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 602/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:15.120994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:15.198833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:15.267626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf50539-3eba-4be8-9870-74f9c61a181c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:15.330615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:15.332091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:15.332585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:15.333125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:15.357459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:15.360359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:15.393151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:15.395493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:15.397693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:15.397693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:15.397693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:15.397693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:15.397693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 608/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:15.405223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:15.605284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:15.609247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99aaa3-33e8-47b8-bf29-e51ff89e4b16 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:15.675882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:15.676843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:15.677229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:15.677555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:15.678478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:15.678877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:15.681075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:15.703546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 603/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:18:15.710112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:15.865246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709196c-3ee9-458a-a131-39ca91fa017a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:15.868215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:15.925825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:15.926902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:15.927315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:15.927742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:15.942990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:15.946306 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:18:15.977358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:15.980248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:15.982756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:15.989315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:15.990495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 609/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:16.134637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7296d-3889-49b5-b533-62b92df866d8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:16.184877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:16.185659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:16.186013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:16.186329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:16.187354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:16.187723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:16.189483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:16.214047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 604/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Jun 16 18:18:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:16.220859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:16.403527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f039a2-3981-4609-aa66-0e11ba962a39 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:16.468131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:16.469218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:16.469657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:16.470080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:16.534026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:16.602881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:16.605008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:16.748129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 610/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:18:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:18:16.755052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:16.783800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:16.893352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20caab-8acc-49ea-8f28-5172493229ff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:16.939532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:16.940940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:16.941261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:16.941688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:16.960916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:16.963908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:16.989442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:16.990717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:16.995094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:16.995094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:16.995094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:16.995094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:16.995094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 605/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:16.999764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:17.086105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:17.163674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb8890-5ebc-4a5e-b367-809d94a64fd7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:17.211815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:17.213253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:17.213932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:17.214485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:17.215685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:17.216407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:17.218526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:17.236476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 611/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:18:17.246459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:17.341272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:17.382581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea1e26-7549-4d7b-8c7d-ec93afa0b840 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:17.430749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:17.431682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:17.431975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:17.433960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:17.440302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:17.441714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:18:17.464121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:17.465466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:17.466382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 606/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:18:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:17.473202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:17.602238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4f970-845b-40f1-a91b-35bbd4967a77 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:17.639040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:17.664904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:17.666178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:17.666505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:17.666774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:17.667672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:17.667994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:17.669509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1be68e-3254-4ab4-a718-50dd657b1e8b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:17.686542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 612/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:17.694844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:17.828760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cb13d-3cd4-4ed4-b535-bcb52984fc7c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:17.865006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:17.873723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:17.874590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:17.874977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:17.875363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:17.876277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=cad094d850d946d18b683bfc04c6f3e2) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:17.876602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:17.878091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826b26a-c07f-41f3-bf8a-0213ca14672f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:17.885656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 607/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:18:17 2022] GET /identity/v3/role_assignments?user.id=cad094d850d946d18b683bfc04c6f3e2&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:20.374093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ecd4-b236-486a-8272-b0c0c75c9906 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:20.374775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ecd4-b236-486a-8272-b0c0c75c9906 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:20.374775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ecd4-b236-486a-8272-b0c0c75c9906 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:20.376755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 613/1220] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:20.386352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:20.387074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:20.387074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:20.410287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:20.412095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:20.510454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 608/1221] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:20.552747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5ecd4-b236-486a-8272-b0c0c75c9906 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:20.693367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5ecd4-b236-486a-8272-b0c0c75c9906 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:20.734949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:20.735800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:20.736109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:20.736445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:20.749103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:18:20.759430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:20.761327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:20.762770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 614/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:18:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:20.767039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:20.924472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a17a5-ee2d-493f-b2c3-3ab2f005993d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:20.960164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:20.978290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:20.979335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:20.979518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:20.979781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:20.980616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:20.980991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:20.982751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:21.000207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 609/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:18:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:21.001797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:21.057739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:21.140590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b545e-2b8c-4951-9378-f44dc5e1a93e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:21.180352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:21.181291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:21.181694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:21.181871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:21.197344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:21.199041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:21.220588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:21.221705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:21.222637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:21.222637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:21.222637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:21.222637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:21.222637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 615/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:21.228176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:21.349850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d275-f605-4b05-9905-09d35acb8598 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:21.388250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:21.389283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:21.389283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:21.389605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:21.390183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:21.390494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:21.391869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:21.411875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 610/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:21.420403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:21.550414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:21.560038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025e14b-5f29-498e-b7cf-d3561b6d143b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:21.599323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:21.600252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:21.600658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:21.601121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:21.610834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:21.612324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:18:21.628499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:21.631007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:21.632433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 616/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:21.638472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:21.697016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:21.776274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977739-d1e1-4024-8821-69ea6465cbe1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:21.828478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:21.829185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:21.829471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:21.829751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:21.830630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:21.830948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:21.832582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:21.857097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 611/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Jun 16 18:18:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:21.863233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:22.018259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b91f-0e76-4df3-86b7-a4ca8f3b20c6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:22.063255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:22.063982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:22.064292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:22.064567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:22.082022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:22.083983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:22.111281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:22.112474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:22.113587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:22.113587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:22.113587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:22.113587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:22.113587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 617/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:22.118698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:22.233067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275a965-18ee-47b7-8a93-02ac5f0c65e9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:22.236376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:22.276669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:22.277366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:22.277753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:22.277938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:22.278680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:22.278991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:22.280397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:22.294108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 612/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:22.300118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:22.312136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:22.428385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab848a9-4cc4-46e1-81b5-11a46b7782d4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:22.470924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:22.471729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:22.472016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:22.472323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:22.479595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:22.481103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:18:22.496896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:22.499113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:22.500645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 618/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:22.505500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:22.618376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44183e13-e27e-47c6-b3ae-f3fcbda3a11a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:22.656318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:22.657067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:22.657356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:22.657636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:22.658438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:22.658821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:22.660218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1aa20-4020-4031-b8f0-d5b47f29e4e1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:22.677337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 613/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:18:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:22.690556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:22.768035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:22.810627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144599d9-3e1e-4159-b10a-c3dffb499464 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:22.849378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e011a-b7ba-48b3-b039-e5275de46b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:22.850287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e011a-b7ba-48b3-b039-e5275de46b8e None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:22.850572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e011a-b7ba-48b3-b039-e5275de46b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:22.850844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e011a-b7ba-48b3-b039-e5275de46b8e None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/cad094d850d946d18b683bfc04c6f3e2/roles/aa67c6c6ff7c4b9080495cfdb578693b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:22.866920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e011a-b7ba-48b3-b039-e5275de46b8e None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=cad094d850d946d18b683bfc04c6f3e2, role_id=aa67c6c6ff7c4b9080495cfdb578693b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:22.868722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e011a-b7ba-48b3-b039-e5275de46b8e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:22.883125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 619/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:18:22 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/cad094d850d946d18b683bfc04c6f3e2/roles/aa67c6c6ff7c4b9080495cfdb578693b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:25.301100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa44e4-6649-4956-b901-d065336161ad None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:25.301753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa44e4-6649-4956-b901-d065336161ad None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:25.301753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa44e4-6649-4956-b901-d065336161ad None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:25.303013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 614/1233] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:25.310066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:25.310843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:25.310968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:25.337255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:25.339551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:25.440607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 620/1234] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:25.468478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa44e4-6649-4956-b901-d065336161ad None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:25.501417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:25.608523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa44e4-6649-4956-b901-d065336161ad None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:25.656741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:25.657555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:25.657836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:25.658109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:25.670623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:18:25.678696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:25.680037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:25.681063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 615/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Jun 16 18:18:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:25.687830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:25.743205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:25.814484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385caae0-03dd-4ca0-9935-f60ba052e499 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:25.871043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:25.872028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:25.872558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:25.873019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:25.874272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:25.874626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:25.876624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:25.890364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 621/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Jun 16 18:18:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:25.899417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:26.059440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:26.088643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4f4c-9b50-4f92-a294-e230cf2768ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:26.131235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:26.131934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:26.132279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:26.133071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:26.148947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:26.149741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:26.171832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:26.173560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:26.175121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:26.175121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:26.175121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:26.175121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:26.175121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 616/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:26.179695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:26.301876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4664-ae6e-46fe-b209-1ae087e0fc7e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:26.316589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:26.348267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:26.349148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:26.349570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:26.349948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:26.350928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:26.351458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:26.353177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:26.376838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 622/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:26.380110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:26.529146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1da49-6db1-4b7f-91f5-02bfd28ee28e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:26.587530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:26.588469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:26.588786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:26.589065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:26.592815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:26.600332 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:26.603038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:18:26.625899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:26.627040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:26.628920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 617/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:26.634896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:26.775883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb8fce-36e2-4898-a679-e2dfebe6ea19 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:26.824293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:26.825218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:26.825665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:26.825947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:26.827082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:26.827424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:26.829102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:26.854883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 623/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Jun 16 18:18:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:26.860658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:27.021011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4101681-fba7-43d0-88dc-67866b9960a2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:27.061885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:27.062497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:27.062785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:27.063058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:27.151792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:27.159352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:27.160939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:27.313383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 618/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:18:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:18:27.323687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:27.357954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:27.475994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be23a97-96b3-49cc-9c97-b9a2b7cad7c6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:27.534671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:27.536687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:27.537217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:27.537653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:27.559904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:27.561402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:27.588452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:27.590265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:27.591896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:27.591896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:27.591896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:27.591896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:27.591896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 624/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:27.596458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:27.721118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:27.728568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe5d2b-c3c7-4528-b385-ca5fe01c5295 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:27.767470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:27.768177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:27.768459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:27.768731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:27.769538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:27.769847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:27.771516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:27.785354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 619/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:27.792895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:27.916445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:27.948536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bd460-3173-4c99-afb3-a1eecdd6733a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:28.002668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:28.003720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:28.004008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:28.004392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:28.014755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:28.016604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:18:28.038612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:28.042066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:28.043162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 625/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:28.046804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:28.170690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4e5c9-9029-41b3-8c7f-6367fea70dae None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:28.211379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:28.212621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:28.212910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:28.213192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:28.214267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:28.214597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:28.216415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87d40f-95cb-405a-a3a7-7f9aae9d86db None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:28.232394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 620/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:28.238739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:28.368490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad9ce2-1f06-4d1f-b45a-402ca1ecad0a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:28.419600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:28.420294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:28.420810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:28.420969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:28.421907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] RBAC: Adding query filter params (role.id=aa67c6c6ff7c4b9080495cfdb578693b, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=cad094d850d946d18b683bfc04c6f3e2) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:28.422349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:28.424062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394dd7d-641d-423b-bf59-e8d505e31fe8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:28.432542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 626/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:18:28 2022] GET /identity/v3/role_assignments?user.id=cad094d850d946d18b683bfc04c6f3e2&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=aa67c6c6ff7c4b9080495cfdb578693b => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:18:30.926228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9a7bd-70a5-4c30-b4ab-a8ce668099d4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:30.926863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9a7bd-70a5-4c30-b4ab-a8ce668099d4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:30.926863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9a7bd-70a5-4c30-b4ab-a8ce668099d4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:30.928780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 621/1247] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:18:30.935519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:30.936107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:30.936333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:30.943384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:30.965818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:30.967254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:31.087091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 627/1248] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:31.113112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9a7bd-70a5-4c30-b4ab-a8ce668099d4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:31.140236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:31.284911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9a7bd-70a5-4c30-b4ab-a8ce668099d4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:31.332654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:31.333503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:31.333779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:31.334052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:31.347107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:18:31.355949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:31.357408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:31.359168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 622/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:31.365779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:31.516391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b80d4-d425-4471-9829-d43627be69ed None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:31.554817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:31.562492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:31.563186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:31.563471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:31.563745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:31.564607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:31.565196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:31.566709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:31.578098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 628/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:31.585041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:31.693706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:31.725573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153159b-1e15-4ae1-b292-f9803acc2f95 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:31.775578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:31.776486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:31.776895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:31.777018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:31.791985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:31.793618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:31.818255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:31.820998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:31.823320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:31.823320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:31.823320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:31.823320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:31.823320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 623/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:18:31.827039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:31.966483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bf2b1-b14e-43d8-b6dc-581d5952ddc9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:32.009287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:32.010028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.010310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.010586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.011366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:32.011711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:32.013239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:32.028254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 629/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:32.035545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:32.175426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0e324-12f0-413a-b38b-291a1930fc41 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:32.206733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:32.223675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:32.223675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.223675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.223675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.230494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:32.232174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:18:32.250526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:32.251781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:32.252799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 624/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:32.257938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:32.350722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:32.417885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208ce75-7a34-4b36-84c2-56938ed2f42e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:32.472681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:32.474312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.474886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.475613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.477923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:32.478655 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:32.484770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:32.520929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 630/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Jun 16 18:18:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:32.527315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:32.702372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d60a0e-7cc7-41ff-9b8b-572d150bd535 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:32.749001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:32.749643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.749921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.750199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.812760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:32.827721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:32.829246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:32.957366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 625/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:18:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:18:32.959227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.118915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcf7a5-0f63-42e9-8357-888f53f52497 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.139156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:33.177147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.178582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.179113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.180262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.204140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.207214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:33.244230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:33.245694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:33.247388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:33.247388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:33.247388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:33.247388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:33.247388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 631/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:18:33.251877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.375704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:33.393632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8bfe-56c1-423c-85bc-53a9e11c3d47 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.439267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.439941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.440278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.440564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.441278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:33.441629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.443021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.459244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 626/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:33.467891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.606227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55208ebe-196b-4254-8f0d-7451186a278e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.662056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.663714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.665732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.665732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.676603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.678075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:18:33.694863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:33.696370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:33.698386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 632/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:18:33.703588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.843305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837adf3-3835-48ae-900d-b3905a1d19bd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.886806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.887490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.887780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.888067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.888945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:33.889194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.890650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc355f00-2c57-4e8e-90f1-ce4d227d1472 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.904570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 627/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:33.913642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.960190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:34.099816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb41a9-83ea-4e6b-88b1-7925b7f46be9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.143940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.143940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.150376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.150376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.150376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=cad094d850d946d18b683bfc04c6f3e2) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:34.150376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.150376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233754ea-4929-4488-a725-cdad504b4b3c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.156924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 633/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:18:33 2022] GET /identity/v3/role_assignments?user.id=cad094d850d946d18b683bfc04c6f3e2&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:36.433335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055c2ed-019c-44c7-8c3c-f94db8d0a262 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.433904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055c2ed-019c-44c7-8c3c-f94db8d0a262 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.433904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055c2ed-019c-44c7-8c3c-f94db8d0a262 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.435253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 628/1261] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:36.440555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.441061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.441196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.459089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:36.460347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:36.535551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 634/1262] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:18:36.560759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055c2ed-019c-44c7-8c3c-f94db8d0a262 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.582008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:36.685641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055c2ed-019c-44c7-8c3c-f94db8d0a262 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.744374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.745168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.745448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.745720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.756501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:18:36.764469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:36.765610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:36.766498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 629/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:18:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:36.772098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.791399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:36.877134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd36095-dab0-4309-a438-daaa16112c59 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.912596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.913266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.913555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.913863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.914622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:36.914933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.916928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.925321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 635/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:36.931551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.022959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.066980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf1eec-bb67-4112-96bd-fa0e80e4f00c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.115312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.116755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.117849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.118316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.141803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.143315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:37.163218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:37.165157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:37.166585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:37.166585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:37.166585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:37.166585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:37.166585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 630/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:37.171328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.263681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.284817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd10227-447f-4f8a-b770-8e40aebada88 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.321629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.322303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.322584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.322856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.323585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:37.323890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.325292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.337601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 636/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:37.343866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.453186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e24173-0a58-4008-a74a-ee59655b6bb4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.474092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.490461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.491277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.491560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.491867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.500318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.501801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:18:37.517437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:37.518634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:37.519974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 631/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:37.525110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.628475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134910a-f423-4313-8342-52b7fd0473aa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.664882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.665646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.665924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.666460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.667309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:37.667611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.669081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.678041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.690672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 637/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:18:37.699227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.808379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfbe15-4725-4918-8d6e-d8f46ee13410 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.842902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.843700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.843983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.844284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.855847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.857374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:37.874805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:37.875992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:37.877282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:37.877282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:37.877282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:37.877282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:37.877282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 632/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:37.882106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:38.001592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a761b-9943-426b-ace1-4092188592c6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:38.039487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:38.041304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:38.041590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:38.041873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:38.042621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:38.042927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:38.044350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:38.056944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 638/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:38.062626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:38.095386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:38.168980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e22b0-a21d-4255-b561-3db34681d959 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:38.206424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:38.207240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:38.207515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:38.207792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:38.215536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:38.217010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:18:38.232695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:38.233814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:38.235032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 633/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:38.239829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:38.311711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:38.362190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd274ec-532f-4229-9b0e-7278fc75b02a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:38.422658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:38.423733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:38.424284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:38.424798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:38.426388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:38.426970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:38.430644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deb6b8-e5c3-4ab8-a6e6-9991e95fcc8f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:38.454409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 639/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:38.462220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:38.562998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:38.577344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cff8c-a26e-4f87-8b8d-727a2566d4a5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:38.619033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64feb1-c284-415e-82b2-c3a560d5b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:38.619942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64feb1-c284-415e-82b2-c3a560d5b420 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:38.620251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64feb1-c284-415e-82b2-c3a560d5b420 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:38.620529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64feb1-c284-415e-82b2-c3a560d5b420 None admin] PATH_INFO: `/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/cad094d850d946d18b683bfc04c6f3e2/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:38.622510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64feb1-c284-415e-82b2-c3a560d5b420 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fabfeb40ce4613a56f73db68fde3ec, user_id=cad094d850d946d18b683bfc04c6f3e2, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:38.623951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64feb1-c284-415e-82b2-c3a560d5b420 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:38.643084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 634/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Jun 16 18:18:38 2022] PUT /identity/v3/projects/c0fabfeb40ce4613a56f73db68fde3ec/users/cad094d850d946d18b683bfc04c6f3e2/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:40.828721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d03fdd-5967-4356-8211-628e67a385e0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:40.829261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d03fdd-5967-4356-8211-628e67a385e0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:40.829261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d03fdd-5967-4356-8211-628e67a385e0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:40.831458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 640/1274] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:40.838890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:40.839399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:40.839519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:40.859463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:40.861070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:40.938344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 635/1275] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:40.965700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d03fdd-5967-4356-8211-628e67a385e0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.112643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d03fdd-5967-4356-8211-628e67a385e0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.131151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.151230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.152041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.152352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.152627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.165147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:18:41.177569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:41.178870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:41.180606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 641/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Jun 16 18:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:41.185562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.280973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.300998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a9cd2-a15b-4a89-ae2b-27ec3fd05640 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.337588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.338351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.338554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.338830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.339582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:41.339897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.341281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.349759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 636/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:41.355642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.457867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb092f1-3544-4798-8783-5ef3576aca4b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.491264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.492606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.492884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.493176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.504882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.505953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:41.525662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:41.526834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:41.528112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:41.528112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:41.528112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:41.528112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:41.528112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 642/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:41.532666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.635783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f39eae-aa2b-4073-b534-9cd89d05c7aa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.670637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.671304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.671588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.671868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.672617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:41.672930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.674298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.686783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 637/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:41.692701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.726243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.806087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f04b0-a978-4c25-85c1-577e1e004b58 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.841911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.842707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.842984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.843250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.850756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.852261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:18:41.866160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:41.867307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:41.868437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 643/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:41.873150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.877010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.978648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53c9fc-c866-4c6c-be9e-83bab813b86a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.013240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.013918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.014201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.014474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.015308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:42.015617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.017089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.036219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 638/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:42.043456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.135265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.145076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ea4be-8c18-46e2-bca1-6189956707a5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.181699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.182349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.182626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.182900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.254421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.255854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.346750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.358334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 644/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Jun 16 18:18:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:18:42.364154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.429093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.469881 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71765c6b-5dc7-462e-bed2-a59dbc66eb09 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.504633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.505362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.506255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.506255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.517530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.518982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:18:42.537298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:42.538420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:42.540185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:18:42.540185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:18:42.540185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:18:42.540185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:42.540185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 639/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:42.544901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.649021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01157bce-a9fc-41ae-b3e2-aaa39d3bc56b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.684554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.685224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.685508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.685793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.686514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:42.686820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.688225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.700587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 645/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Jun 16 18:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:42.706429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.813350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be15f3-6888-48ba-b311-0d1389957491 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.828533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.850836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.851727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.852015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.852323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.859339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.860824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:18:42.875020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:42.876603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 16 18:18:42.877700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 640/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Jun 16 18:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:42.882150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.935918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.986622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704f73e-5ad3-49da-84c2-0070f1fd8fab None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:43.021868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:43.022531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:43.022814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:43.023110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:43.024259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:43.024577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:43.026008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf2c4c-9d68-4753-be55-0fbc4d6f40b8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:43.038394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 646/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Jun 16 18:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:43.045716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:43.153192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c75bd-27c1-4235-ba03-200b294513c3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:43.190688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:43.191355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:43.191638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:43.191914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:43.192787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] RBAC: Adding query filter params (role.id=bd65b447e4544528b91e8a65a4f8ee54, scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec, user.id=cad094d850d946d18b683bfc04c6f3e2) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:43.193107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:43.194564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01e861-4d0e-4d04-bdc2-c80c3229136c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:43.200831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 641/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Jun 16 18:18:43 2022] GET /identity/v3/role_assignments?user.id=cad094d850d946d18b683bfc04c6f3e2&scope.project.id=c0fabfeb40ce4613a56f73db68fde3ec&role.id=bd65b447e4544528b91e8a65a4f8ee54 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:45.421377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850185e-4c4d-4feb-a994-33f82d8b8fe6 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.421951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850185e-4c4d-4feb-a994-33f82d8b8fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.421951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850185e-4c4d-4feb-a994-33f82d8b8fe6 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.423294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 647/1288] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:45.428768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.429262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.429502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.447308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:45.448563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:45.481226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:45.529514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 642/1289] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:45.553100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850185e-4c4d-4feb-a994-33f82d8b8fe6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.674412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850185e-4c4d-4feb-a994-33f82d8b8fe6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.687407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:45.714480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.715296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.715575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.715851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.716557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.717951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:45.731666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:45.733092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:45.734190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 648/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:45.739287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.842378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5644d0-ff5d-42b4-a155-206091c6b8d5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.876327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.876999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.877276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.877546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.878249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:45.878553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.879908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640a90-e02b-4aaf-8823-190cd03f5319 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.884932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 643/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:45.890714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:46.002335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53afd61-b353-452d-823c-a7dc11285224 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:46.041024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:46.041938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:46.042213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:46.042478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:46.043189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:46.043485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:46.044878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfcfd4-dcae-46e3-bc77-786e5b885015 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:46.049536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 649/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:48.235610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f1834-a02c-45f6-b367-afbcd92882d2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:48.236183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f1834-a02c-45f6-b367-afbcd92882d2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:48.236183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f1834-a02c-45f6-b367-afbcd92882d2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:48.237582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 644/1293] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:48.243852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a894a-7d2a-4848-8af1-664e40fc6034 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:48.244344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a894a-7d2a-4848-8af1-664e40fc6034 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:48.244438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a894a-7d2a-4848-8af1-664e40fc6034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:48.262539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f79a894a-7d2a-4848-8af1-664e40fc6034 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:48.263739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f79a894a-7d2a-4848-8af1-664e40fc6034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:48.295087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:48.335234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 650/1294] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:48.358550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f1834-a02c-45f6-b367-afbcd92882d2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:48.390939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:48.466783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f1834-a02c-45f6-b367-afbcd92882d2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:48.501738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e8a54-6c5f-454d-b5c3-2b66b0277613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:48.502353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e8a54-6c5f-454d-b5c3-2b66b0277613 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:48.502629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e8a54-6c5f-454d-b5c3-2b66b0277613 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:48.502897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e8a54-6c5f-454d-b5c3-2b66b0277613 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:48.503635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e8a54-6c5f-454d-b5c3-2b66b0277613 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:48.505143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e8a54-6c5f-454d-b5c3-2b66b0277613 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:48.522415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 645/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Jun 16 18:18:48 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:50.755744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc63ed7-d365-4430-be6b-2e5f20e4b710 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.756358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc63ed7-d365-4430-be6b-2e5f20e4b710 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.756358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc63ed7-d365-4430-be6b-2e5f20e4b710 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.758422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 651/1296] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:50.763879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.764353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.764493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.782027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:50.783242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:50.861419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 646/1297] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:18:50.884428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc63ed7-d365-4430-be6b-2e5f20e4b710 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:51.000682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc63ed7-d365-4430-be6b-2e5f20e4b710 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:51.047051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:51.047783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:51.048087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:51.048369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] PATH_INFO: `/v3/services/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:51.049081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:51.050459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:51.062833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:51.063845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:51.064893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 652/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:18:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:51.071241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:51.089684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:51.174819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25709027-92c6-4a20-b05b-4dd6f9149d61 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:51.208807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:51.209481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:51.209768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:51.210048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:51.211339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:51.211339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:51.213004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:51.218344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 647/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Jun 16 18:18:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:51.224366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:51.256918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:51.329094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3aa2-7fdb-468a-8d65-a69bb52cdaba None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:51.363178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:51.363856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:51.364204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:51.364482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:51.365280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] RBAC: Adding query filter params (interface=public, service_id=989c9533dde44fc98ade4531c249bd53, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:51.365597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:51.367321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385a704-34b8-40ce-9d7c-2561c9be5a2c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:51.372667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 653/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Jun 16 18:18:51 2022] GET /identity/v3/endpoints?service_id=989c9533dde44fc98ade4531c249bd53&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:51.378578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:51.464565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:51.484202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3d0d2-9b6e-4c6a-a913-a8eb94460a70 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:51.519599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4cd4b2-c811-4466-9dde-717d221e4325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:51.520299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cd4b2-c811-4466-9dde-717d221e4325 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:51.520589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cd4b2-c811-4466-9dde-717d221e4325 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:51.520868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cd4b2-c811-4466-9dde-717d221e4325 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:51.521568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4cd4b2-c811-4466-9dde-717d221e4325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:51.522956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4cd4b2-c811-4466-9dde-717d221e4325 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:51.530321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 648/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Jun 16 18:18:51 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:53.701668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cd765-7634-4c90-bcd5-9d0c68d74ee8 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:53.702274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cd765-7634-4c90-bcd5-9d0c68d74ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:53.702274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cd765-7634-4c90-bcd5-9d0c68d74ee8 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:53.703576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 654/1302] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:18:53.709077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:53.709550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:53.709645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:53.727660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:53.728919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:53.797234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 649/1303] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:18:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:18:53.821145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cd765-7634-4c90-bcd5-9d0c68d74ee8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:53.922634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:53.931051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cd765-7634-4c90-bcd5-9d0c68d74ee8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:53.970475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:53.971400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:53.971720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:53.972078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:53.972877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:53.974534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:18:53.988868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:18:53.990438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 16 18:18:53.991702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 655/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Jun 16 18:18:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:18:53.998382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:54.080117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:54.104208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463505d5-0715-4de8-b56b-9df28a626c5d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:54.139133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:54.139855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:54.140205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:54.140525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:54.141284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:54.141631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:54.143061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f047a51-996a-407f-9389-e644c45dbcb0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:54.149583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 650/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Jun 16 18:18:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:54.156281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:54.261824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fff75-c00b-42aa-9b64-8a6f4ad0d2d0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:54.295875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946a536-d1a1-4096-83e4-30e764909b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:54.296566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a536-d1a1-4096-83e4-30e764909b2e None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:54.296840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a536-d1a1-4096-83e4-30e764909b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:54.297109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a536-d1a1-4096-83e4-30e764909b2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:54.297850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946a536-d1a1-4096-83e4-30e764909b2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:54.299197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946a536-d1a1-4096-83e4-30e764909b2e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:54.317865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 656/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Jun 16 18:18:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:04.146047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=30 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:04.313431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:00.789589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727333b-ea97-4a4d-aa9b-4c5e43e2f799 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:00.790072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727333b-ea97-4a4d-aa9b-4c5e43e2f799 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:00.790419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727333b-ea97-4a4d-aa9b-4c5e43e2f799 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:00.792405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 651/1307] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:00.798536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c4cb8-0399-499d-aea8-2271e88dca07 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:00.798536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c4cb8-0399-499d-aea8-2271e88dca07 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:00.798536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c4cb8-0399-499d-aea8-2271e88dca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:00.946886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-178c4cb8-0399-499d-aea8-2271e88dca07 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:00.949817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-178c4cb8-0399-499d-aea8-2271e88dca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:01.107768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 657/1308] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:20:01.183695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674f52-e4d5-4dbd-853b-f54f074f2c6f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:01.184101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674f52-e4d5-4dbd-853b-f54f074f2c6f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:01.184321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674f52-e4d5-4dbd-853b-f54f074f2c6f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:01.185973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 652/1309] 10.222.0.30 () {54 vars in 964 bytes} [Thu Jun 16 18:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:01.192328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c6613-240b-42d6-92f7-dca94088f9ca None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:01.192630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c6613-240b-42d6-92f7-dca94088f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:01.192915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c6613-240b-42d6-92f7-dca94088f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:01.285064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0c0c6613-240b-42d6-92f7-dca94088f9ca None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:01.286405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0c0c6613-240b-42d6-92f7-dca94088f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:01.311222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:01.380092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 658/1310] 10.222.0.30 () {58 vars in 1061 bytes} [Thu Jun 16 18:20:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:01.390510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674f52-e4d5-4dbd-853b-f54f074f2c6f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:01.392650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5b674f52-e4d5-4dbd-853b-f54f074f2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:01.535673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674f52-e4d5-4dbd-853b-f54f074f2c6f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:01.637178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:01.637760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:01.638185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:01.638614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:01.640395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:01.722391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:01.794172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:01.795616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97f6f-4007-434d-8b4e-c65ab36aee22 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:01.815989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 653/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:04.904392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b83b1a-f7b3-4233-b714-e4eda0ea343a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:04.905226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b83b1a-f7b3-4233-b714-e4eda0ea343a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:04.905382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b83b1a-f7b3-4233-b714-e4eda0ea343a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:04.908339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 659/1312] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:04.914879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28b3d-b1e0-43ad-99af-4a5743ee7624 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:04.915741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28b3d-b1e0-43ad-99af-4a5743ee7624 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:04.915931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28b3d-b1e0-43ad-99af-4a5743ee7624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:05.020143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9bc28b3d-b1e0-43ad-99af-4a5743ee7624 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:05.021972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9bc28b3d-b1e0-43ad-99af-4a5743ee7624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:05.136183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 654/1313] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:20:05.191323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7601ad-d986-469f-9eb0-74d963eb3e2e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:05.191757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7601ad-d986-469f-9eb0-74d963eb3e2e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:05.192234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7601ad-d986-469f-9eb0-74d963eb3e2e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:05.195732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 660/1314] 10.222.0.30 () {54 vars in 964 bytes} [Thu Jun 16 18:20:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:05.202605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4008592-f91e-4d54-acc5-41687eb7e1a1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:05.202893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4008592-f91e-4d54-acc5-41687eb7e1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:05.203176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4008592-f91e-4d54-acc5-41687eb7e1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:05.214321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:05.296647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d4008592-f91e-4d54-acc5-41687eb7e1a1 None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:05.297762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d4008592-f91e-4d54-acc5-41687eb7e1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:05.404942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 655/1315] 10.222.0.30 () {58 vars in 1061 bytes} [Thu Jun 16 18:20:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:05.415065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7601ad-d986-469f-9eb0-74d963eb3e2e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:05.416648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8f7601ad-d986-469f-9eb0-74d963eb3e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:05.578235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7601ad-d986-469f-9eb0-74d963eb3e2e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:05.680669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:05.683994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:05.685050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:05.685481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:05.686016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:05.687881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:05.821518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:05.822969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b25408-e3b5-4bb1-a606-d1da33e3f5ae service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:05.843954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 661/1316] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:08.601323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61473a2-d99f-4d9b-97cc-24f8da0d8a19 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:08.601984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61473a2-d99f-4d9b-97cc-24f8da0d8a19 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:08.601984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61473a2-d99f-4d9b-97cc-24f8da0d8a19 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:08.603264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 656/1317] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:08.608539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b780e2-90d9-4de3-8cd5-68b08fc5ff5c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:08.608977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b780e2-90d9-4de3-8cd5-68b08fc5ff5c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:08.609079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b780e2-90d9-4de3-8cd5-68b08fc5ff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:08.685052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-27b780e2-90d9-4de3-8cd5-68b08fc5ff5c None None] MFA Rules not processed for user `8593faba1d9d4e718339612ca5bec325`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:08.686320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-27b780e2-90d9-4de3-8cd5-68b08fc5ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:08.744316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:08.791624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 662/1318] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:08.832520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61473a2-d99f-4d9b-97cc-24f8da0d8a19 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:08.835305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a61473a2-d99f-4d9b-97cc-24f8da0d8a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:08.946640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61473a2-d99f-4d9b-97cc-24f8da0d8a19 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:08.949570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:09.026336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:09.026987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:09.027267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:09.027558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:09.029253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:09.148597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:09.150049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90beb-7b24-4915-a465-4a15af9a1f71 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:09.170381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 657/1319] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:11.971489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c027f9-a216-4a4d-be3b-d6865294f7ef None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:11.972140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c027f9-a216-4a4d-be3b-d6865294f7ef None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:11.972468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c027f9-a216-4a4d-be3b-d6865294f7ef None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:11.974000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 663/1320] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:11.980007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b839b5-0a3a-4873-8eae-262c800e577a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:11.980488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b839b5-0a3a-4873-8eae-262c800e577a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:11.980599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b839b5-0a3a-4873-8eae-262c800e577a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:12.020460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:12.047365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-74b839b5-0a3a-4873-8eae-262c800e577a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:12.048635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-74b839b5-0a3a-4873-8eae-262c800e577a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:12.145829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 658/1321] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:12.185545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c027f9-a216-4a4d-be3b-d6865294f7ef None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:12.187111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-65c027f9-a216-4a4d-be3b-d6865294f7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:12.295898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c027f9-a216-4a4d-be3b-d6865294f7ef None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:12.373852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:12.374506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:12.374791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:12.375068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:12.376767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:12.385804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:12.495259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:12.496721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ae526-c3f8-4589-bbf6-53218b313846 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:12.516432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 664/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:15.451711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3300f1d-245e-4cd3-9024-84551ad00ca5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:15.452296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3300f1d-245e-4cd3-9024-84551ad00ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:15.452296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3300f1d-245e-4cd3-9024-84551ad00ca5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:15.453734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 659/1323] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:15.460845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c560a-dd8f-4741-85db-ad83c82607a1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:15.461444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c560a-dd8f-4741-85db-ad83c82607a1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:15.461628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c560a-dd8f-4741-85db-ad83c82607a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:15.541392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-703c560a-dd8f-4741-85db-ad83c82607a1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:15.543276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-703c560a-dd8f-4741-85db-ad83c82607a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:15.612482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:15.659606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 665/1324] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:15.697023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3300f1d-245e-4cd3-9024-84551ad00ca5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:15.698689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c3300f1d-245e-4cd3-9024-84551ad00ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:15.814992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3300f1d-245e-4cd3-9024-84551ad00ca5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:15.829128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:15.896837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:15.897397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:15.897679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:15.898153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:15.899699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:16.016759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:16.018217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916695-76a7-4ba4-afd5-0801fe9034e6 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:16.038281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 660/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:18.313660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cfb06-ffb7-4462-b363-d382fa09c5f4 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:18.314270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cfb06-ffb7-4462-b363-d382fa09c5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:18.314270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cfb06-ffb7-4462-b363-d382fa09c5f4 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:18.315744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 666/1326] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:18.322058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4bc5-8fb2-4ef2-9d2d-f10ec5320d05 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:18.322579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4bc5-8fb2-4ef2-9d2d-f10ec5320d05 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:18.322687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4bc5-8fb2-4ef2-9d2d-f10ec5320d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:18.372233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:18.392652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1e2b4bc5-8fb2-4ef2-9d2d-f10ec5320d05 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:18.393911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e2b4bc5-8fb2-4ef2-9d2d-f10ec5320d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:18.499585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 661/1327] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:18.536793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cfb06-ffb7-4462-b363-d382fa09c5f4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:18.538343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cd2cfb06-ffb7-4462-b363-d382fa09c5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:18.649368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cfb06-ffb7-4462-b363-d382fa09c5f4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:18.732999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:18.733743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:18.734112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:18.734388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:18.736202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:18.754194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:18.852199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:18.854135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6b715-f1b5-49ce-93e0-d1857b8dace0 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:18.873701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 667/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:21.106612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee195e90-6d85-4030-aa3b-1c0c082c2880 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:21.107197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee195e90-6d85-4030-aa3b-1c0c082c2880 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:21.107197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee195e90-6d85-4030-aa3b-1c0c082c2880 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:21.108610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 662/1329] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:21.114260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f589-7338-432e-8f6f-904f410bcf19 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:21.114880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f589-7338-432e-8f6f-904f410bcf19 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:21.114976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f589-7338-432e-8f6f-904f410bcf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:21.189179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9410f589-7338-432e-8f6f-904f410bcf19 None None] MFA Rules not processed for user `8593faba1d9d4e718339612ca5bec325`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:21.190507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9410f589-7338-432e-8f6f-904f410bcf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:21.262913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:21.300394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 668/1330] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:21.340030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee195e90-6d85-4030-aa3b-1c0c082c2880 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:21.341637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ee195e90-6d85-4030-aa3b-1c0c082c2880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:21.484718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee195e90-6d85-4030-aa3b-1c0c082c2880 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:21.507943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:21.565558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:21.566226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:21.566517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:21.566797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:21.569095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:21.690984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:21.692420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae4f91-633b-4560-9bd0-f2a4499113bd service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:21.713932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 663/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:24.342704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee57e-9d88-492f-be47-8b63b60762fb None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:24.343255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee57e-9d88-492f-be47-8b63b60762fb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:24.343255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee57e-9d88-492f-be47-8b63b60762fb None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:24.344708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 669/1332] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:24.350641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3e938-6e84-4909-b592-aff442c60e68 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:24.351156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3e938-6e84-4909-b592-aff442c60e68 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:24.351254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3e938-6e84-4909-b592-aff442c60e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:24.418889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:24.421703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6bd3e938-6e84-4909-b592-aff442c60e68 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:24.423543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6bd3e938-6e84-4909-b592-aff442c60e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:24.526259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 664/1333] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:24.563454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee57e-9d88-492f-be47-8b63b60762fb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:24.565053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-383ee57e-9d88-492f-be47-8b63b60762fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:24.672591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee57e-9d88-492f-be47-8b63b60762fb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:24.748879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:24.749544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:24.749871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:24.750104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:24.752072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:24.779907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:24.872472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:24.873907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05396d7-8404-4290-baf4-c4096a8cb76b service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:24.898056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 670/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:27.161582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487ef07-32ad-4a4b-9f3d-937a83a9410c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:27.162271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487ef07-32ad-4a4b-9f3d-937a83a9410c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:27.162373 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487ef07-32ad-4a4b-9f3d-937a83a9410c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:27.163802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 665/1335] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:27.169774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd4898-cfe4-4b8c-a559-4ffe1ee8b476 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:27.170264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd4898-cfe4-4b8c-a559-4ffe1ee8b476 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:27.170376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd4898-cfe4-4b8c-a559-4ffe1ee8b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:27.241235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4ddd4898-cfe4-4b8c-a559-4ffe1ee8b476 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:27.242461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ddd4898-cfe4-4b8c-a559-4ffe1ee8b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:27.323412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:27.355162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 671/1336] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:27.397184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487ef07-32ad-4a4b-9f3d-937a83a9410c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:27.398758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a487ef07-32ad-4a4b-9f3d-937a83a9410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:27.508016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487ef07-32ad-4a4b-9f3d-937a83a9410c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:27.538128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:27.585329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:27.585991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:27.586275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:27.586551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:27.588237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:27.712611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:27.714055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05ab3c-1b31-42e0-bf24-a8a5a29bd201 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:27.732598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 666/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:30.368204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c351a2-46cd-4f5e-9870-f1f23f4016bc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:30.368748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c351a2-46cd-4f5e-9870-f1f23f4016bc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:30.368748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c351a2-46cd-4f5e-9870-f1f23f4016bc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:30.370166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 672/1338] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:30.375885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377320cb-0856-4082-b94a-d195ed8cf9d3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:30.376937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377320cb-0856-4082-b94a-d195ed8cf9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:30.377232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377320cb-0856-4082-b94a-d195ed8cf9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:30.444974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-377320cb-0856-4082-b94a-d195ed8cf9d3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:30.446191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-377320cb-0856-4082-b94a-d195ed8cf9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:30.454680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:30.548059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 667/1339] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:30.584979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c351a2-46cd-4f5e-9870-f1f23f4016bc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:30.586522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-14c351a2-46cd-4f5e-9870-f1f23f4016bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:30.694623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c351a2-46cd-4f5e-9870-f1f23f4016bc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:30.771676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:30.772360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:30.772645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:30.772919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:30.774602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:30.813290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:30.898097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:30.899548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d825243-9bd5-4504-b6ec-283a90b81b4f service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:30.919746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 673/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:20:35.244955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa6fbd-82e8-455c-b2e8-fe93831071e2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:35.245559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa6fbd-82e8-455c-b2e8-fe93831071e2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:35.245664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa6fbd-82e8-455c-b2e8-fe93831071e2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:35.247139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 668/1341] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:35.252879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438919ab-3527-452d-92f4-e2bbbb3f02db None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:35.253408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438919ab-3527-452d-92f4-e2bbbb3f02db None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:35.253408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438919ab-3527-452d-92f4-e2bbbb3f02db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:35.332172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-438919ab-3527-452d-92f4-e2bbbb3f02db None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:35.333812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-438919ab-3527-452d-92f4-e2bbbb3f02db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:35.443266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:35.458088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 674/1342] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:35.497480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa6fbd-82e8-455c-b2e8-fe93831071e2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:35.499631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e8aa6fbd-82e8-455c-b2e8-fe93831071e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:35.609264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa6fbd-82e8-455c-b2e8-fe93831071e2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:35.613103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd2b14-8f2e-4461-ad16-73d6419b9b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:35.613765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd2b14-8f2e-4461-ad16-73d6419b9b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:35.614051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd2b14-8f2e-4461-ad16-73d6419b9b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:35.614329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd2b14-8f2e-4461-ad16-73d6419b9b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:35.623892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd2b14-8f2e-4461-ad16-73d6419b9b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:35.625379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd2b14-8f2e-4461-ad16-73d6419b9b55 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:35.636027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 669/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:38.484530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272821c-a538-41c5-ac82-20439a8d2f5b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:38.485147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272821c-a538-41c5-ac82-20439a8d2f5b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:38.485147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272821c-a538-41c5-ac82-20439a8d2f5b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:38.487151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 675/1344] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:38.493769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5934c-b185-4426-87e6-1225c58ed7cd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:38.494419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5934c-b185-4426-87e6-1225c58ed7cd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:38.494419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5934c-b185-4426-87e6-1225c58ed7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:38.528381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-79c5934c-b185-4426-87e6-1225c58ed7cd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:38.529628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-79c5934c-b185-4426-87e6-1225c58ed7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:38.561809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:38.623312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 670/1345] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:38.661727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272821c-a538-41c5-ac82-20439a8d2f5b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:38.673173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272821c-a538-41c5-ac82-20439a8d2f5b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:38.677206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c43c87-fa13-4a1a-96b2-9fc17d05f292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:38.677886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43c87-fa13-4a1a-96b2-9fc17d05f292 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:38.678172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43c87-fa13-4a1a-96b2-9fc17d05f292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:38.678449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43c87-fa13-4a1a-96b2-9fc17d05f292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:38.688687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c43c87-fa13-4a1a-96b2-9fc17d05f292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:38.690437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c43c87-fa13-4a1a-96b2-9fc17d05f292 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:38.702353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 676/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:43.294830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886ee8-6923-4058-8f07-4a18f0df94a4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:43.295375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886ee8-6923-4058-8f07-4a18f0df94a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:43.295375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886ee8-6923-4058-8f07-4a18f0df94a4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:43.296868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 671/1347] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:43.302143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255e778-0ccf-4890-a16b-8ed72375a99c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:43.302674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255e778-0ccf-4890-a16b-8ed72375a99c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:43.302836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255e778-0ccf-4890-a16b-8ed72375a99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:43.321352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5255e778-0ccf-4890-a16b-8ed72375a99c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:43.322562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5255e778-0ccf-4890-a16b-8ed72375a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:43.331627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 677/1348] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:43.370403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886ee8-6923-4058-8f07-4a18f0df94a4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:43.390258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886ee8-6923-4058-8f07-4a18f0df94a4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:43.413607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8a752-5487-43cd-af67-53e841053c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:43.414370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8a752-5487-43cd-af67-53e841053c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:43.414654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8a752-5487-43cd-af67-53e841053c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:43.415176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8a752-5487-43cd-af67-53e841053c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:43.428367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8a752-5487-43cd-af67-53e841053c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:43.429819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8a752-5487-43cd-af67-53e841053c9a service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:43.439658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 672/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:48.993007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97850fc-5d67-4eeb-a843-b68b89d901e7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:48.993485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97850fc-5d67-4eeb-a843-b68b89d901e7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:48.993806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97850fc-5d67-4eeb-a843-b68b89d901e7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:48.995534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 678/1350] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:49.001949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4e17-8f58-4af2-849c-c706cff58f56 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:49.003264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4e17-8f58-4af2-849c-c706cff58f56 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:49.006002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4e17-8f58-4af2-849c-c706cff58f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:49.127133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-786a4e17-8f58-4af2-849c-c706cff58f56 None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:49.128359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-786a4e17-8f58-4af2-849c-c706cff58f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:49.289466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 673/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:20:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:53.315281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=13 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:54.134150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed1333-9314-41f2-80bb-7906b8a11c3f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:54.134560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed1333-9314-41f2-80bb-7906b8a11c3f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:54.134928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed1333-9314-41f2-80bb-7906b8a11c3f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:54.136449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 679/1352] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:54.142358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8190c-5387-47a1-8df9-e6348cba5ff7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:54.142880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8190c-5387-47a1-8df9-e6348cba5ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:54.142967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8190c-5387-47a1-8df9-e6348cba5ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:54.163144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-21c8190c-5387-47a1-8df9-e6348cba5ff7 None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:54.164393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21c8190c-5387-47a1-8df9-e6348cba5ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:54.173863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 674/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:20:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:55.614111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c40db-2bec-40d9-a0af-d70f979d904d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:55.614768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c40db-2bec-40d9-a0af-d70f979d904d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:55.614768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c40db-2bec-40d9-a0af-d70f979d904d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:55.616441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 680/1354] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:55.621715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb30e3-098e-4933-ae1a-c18073dbd92b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:55.622222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb30e3-098e-4933-ae1a-c18073dbd92b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:55.622555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb30e3-098e-4933-ae1a-c18073dbd92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:55.641105 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0ffb30e3-098e-4933-ae1a-c18073dbd92b None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:55.642366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0ffb30e3-098e-4933-ae1a-c18073dbd92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:55.652287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 675/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:20:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:59.535077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562148dc-e1ab-4236-8194-c3cf430bd306 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:59.536073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562148dc-e1ab-4236-8194-c3cf430bd306 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:59.536469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562148dc-e1ab-4236-8194-c3cf430bd306 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:59.537909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 681/1356] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:20:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:59.543773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42e38-1399-4697-812a-59e970e1948c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:59.544291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42e38-1399-4697-812a-59e970e1948c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:59.544390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42e38-1399-4697-812a-59e970e1948c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:59.564005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d5d42e38-1399-4697-812a-59e970e1948c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:59.565235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d5d42e38-1399-4697-812a-59e970e1948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:59.574404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 676/1357] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:59.631278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3380be-8522-43c4-8107-d9ff4b4af3f8 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:59.631604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3380be-8522-43c4-8107-d9ff4b4af3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:59.631878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3380be-8522-43c4-8107-d9ff4b4af3f8 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:59.633891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 682/1358] 10.222.0.30 () {54 vars in 961 bytes} [Thu Jun 16 18:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:20:59.642185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c6937-975e-49cd-8158-16842114627c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:59.642739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c6937-975e-49cd-8158-16842114627c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:59.643427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c6937-975e-49cd-8158-16842114627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:59.704245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:59.713926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4c4c6937-975e-49cd-8158-16842114627c None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:59.715289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4c4c6937-975e-49cd-8158-16842114627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:59.814078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 677/1359] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Jun 16 18:20:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:59.821916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3380be-8522-43c4-8107-d9ff4b4af3f8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:59.838396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3380be-8522-43c4-8107-d9ff4b4af3f8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:59.844907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f0622-8703-49be-9f09-266268e78836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:59.845994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f0622-8703-49be-9f09-266268e78836 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:59.846524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f0622-8703-49be-9f09-266268e78836 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:59.848159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f0622-8703-49be-9f09-266268e78836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:59.858429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f0622-8703-49be-9f09-266268e78836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:59.860377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f0622-8703-49be-9f09-266268e78836 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:59.872130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 683/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:21:02.301325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a1cc2-c38f-41b9-a8e1-9ecc0b16bdb1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:02.301902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a1cc2-c38f-41b9-a8e1-9ecc0b16bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:02.301902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a1cc2-c38f-41b9-a8e1-9ecc0b16bdb1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:02.303469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 678/1361] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:02.308722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58631-750d-40ea-88f5-af70d2ef36d2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:02.309154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58631-750d-40ea-88f5-af70d2ef36d2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:02.309257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58631-750d-40ea-88f5-af70d2ef36d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:02.329699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1fa58631-750d-40ea-88f5-af70d2ef36d2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:02.331216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1fa58631-750d-40ea-88f5-af70d2ef36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:02.352518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 684/1362] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:02.389830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a1cc2-c38f-41b9-a8e1-9ecc0b16bdb1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:02.408027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a1cc2-c38f-41b9-a8e1-9ecc0b16bdb1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:02.416733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef20ae-f5da-4cdf-bf01-446b9810eee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:02.417973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef20ae-f5da-4cdf-bf01-446b9810eee6 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:02.418625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef20ae-f5da-4cdf-bf01-446b9810eee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:02.419096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef20ae-f5da-4cdf-bf01-446b9810eee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:02.430055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef20ae-f5da-4cdf-bf01-446b9810eee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:02.432145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef20ae-f5da-4cdf-bf01-446b9810eee6 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:02.444861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 679/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:05.028360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e82349-85de-4a49-828e-88f701c7ea05 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:05.028907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e82349-85de-4a49-828e-88f701c7ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:05.028907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e82349-85de-4a49-828e-88f701c7ea05 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:05.030462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 685/1364] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:05.036272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f88a52-7b3d-472e-91c4-9817f7def487 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:05.036796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f88a52-7b3d-472e-91c4-9817f7def487 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:05.036796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f88a52-7b3d-472e-91c4-9817f7def487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:05.055872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f5f88a52-7b3d-472e-91c4-9817f7def487 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:05.057095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f5f88a52-7b3d-472e-91c4-9817f7def487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:05.065318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 680/1365] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:05.110561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc987-387f-4617-bedc-23d022e9234d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:05.110850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc987-387f-4617-bedc-23d022e9234d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:05.111124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc987-387f-4617-bedc-23d022e9234d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:05.113152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 686/1366] 10.222.0.30 () {54 vars in 961 bytes} [Thu Jun 16 18:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:05.118575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf6e4f-768c-4340-b432-95ae8193185b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:05.118860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf6e4f-768c-4340-b432-95ae8193185b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:05.119133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf6e4f-768c-4340-b432-95ae8193185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:05.138555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3fbf6e4f-768c-4340-b432-95ae8193185b None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:05.139744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3fbf6e4f-768c-4340-b432-95ae8193185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:05.150588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 681/1367] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Jun 16 18:21:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:05.158383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc987-387f-4617-bedc-23d022e9234d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:05.174630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc987-387f-4617-bedc-23d022e9234d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:05.181333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb70ca-baae-42d5-b35e-29f36c7e8175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:05.182097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb70ca-baae-42d5-b35e-29f36c7e8175 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:05.182462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb70ca-baae-42d5-b35e-29f36c7e8175 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:05.182806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb70ca-baae-42d5-b35e-29f36c7e8175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:05.192296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb70ca-baae-42d5-b35e-29f36c7e8175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:05.193992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb70ca-baae-42d5-b35e-29f36c7e8175 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:05.203988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 687/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:07.461026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358e6e-f1c0-452a-ad47-e372cbf90479 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:07.461925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358e6e-f1c0-452a-ad47-e372cbf90479 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:07.462367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358e6e-f1c0-452a-ad47-e372cbf90479 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:07.464701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 682/1369] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:07.473808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21bfa-0c62-4b77-8e62-28ef107fb3ae None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:07.474379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21bfa-0c62-4b77-8e62-28ef107fb3ae None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:07.474748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21bfa-0c62-4b77-8e62-28ef107fb3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:07.495973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8ce21bfa-0c62-4b77-8e62-28ef107fb3ae None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:07.497696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8ce21bfa-0c62-4b77-8e62-28ef107fb3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:07.524700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 688/1370] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:07.561613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358e6e-f1c0-452a-ad47-e372cbf90479 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:07.577223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358e6e-f1c0-452a-ad47-e372cbf90479 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:07.584608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c23fc-2e20-4d55-abbe-1eb8fafdd84a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:07.585435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c23fc-2e20-4d55-abbe-1eb8fafdd84a service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:07.585819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c23fc-2e20-4d55-abbe-1eb8fafdd84a service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:07.586201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c23fc-2e20-4d55-abbe-1eb8fafdd84a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:07.596079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c23fc-2e20-4d55-abbe-1eb8fafdd84a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:07.598085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c23fc-2e20-4d55-abbe-1eb8fafdd84a service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:07.610353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 683/1371] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:21:08.199467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e167d2-3530-46e0-85ff-17abbee48659 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:08.199936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e167d2-3530-46e0-85ff-17abbee48659 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:08.200351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e167d2-3530-46e0-85ff-17abbee48659 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:08.201819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 689/1372] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:08.208562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a526-30c2-4ce6-a911-c259a16a56a9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:08.209741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a526-30c2-4ce6-a911-c259a16a56a9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:08.210095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a526-30c2-4ce6-a911-c259a16a56a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:08.231283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3b85a526-30c2-4ce6-a911-c259a16a56a9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:08.232573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3b85a526-30c2-4ce6-a911-c259a16a56a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:08.242841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 684/1373] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:08.544192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2fb4f-91d9-4bf5-92a5-cc83807052be None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:08.544727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2fb4f-91d9-4bf5-92a5-cc83807052be None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:08.544727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2fb4f-91d9-4bf5-92a5-cc83807052be None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:08.546385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 690/1374] 10.222.0.30 () {54 vars in 963 bytes} [Thu Jun 16 18:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:08.552348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96f51-e1eb-40ae-94bd-5a378939b860 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:08.552854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96f51-e1eb-40ae-94bd-5a378939b860 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:08.552933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96f51-e1eb-40ae-94bd-5a378939b860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:08.611353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-02a96f51-e1eb-40ae-94bd-5a378939b860 None None] MFA Rules not processed for user `49b361e969094db296e3fc02eda18dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:08.613641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-02a96f51-e1eb-40ae-94bd-5a378939b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:08.682780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 685/1375] 10.222.0.30 () {58 vars in 1060 bytes} [Thu Jun 16 18:21:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:08.692238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2fb4f-91d9-4bf5-92a5-cc83807052be None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:08.705632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2fb4f-91d9-4bf5-92a5-cc83807052be None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:08.709934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b812c-9d00-434c-9224-42319516a59f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:08.710599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b812c-9d00-434c-9224-42319516a59f service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:08.710884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b812c-9d00-434c-9224-42319516a59f service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:08.711162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b812c-9d00-434c-9224-42319516a59f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:08.723122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b812c-9d00-434c-9224-42319516a59f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:08.726096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b812c-9d00-434c-9224-42319516a59f service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:08.741392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 691/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:21:09.869415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3eb2-6eba-43ea-8448-0ddce316230e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:09.869964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3eb2-6eba-43ea-8448-0ddce316230e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:09.869964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3eb2-6eba-43ea-8448-0ddce316230e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:09.871575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 686/1377] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:09.879934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118944c0-ecdd-48c6-9be9-67885623fc94 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:09.879934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118944c0-ecdd-48c6-9be9-67885623fc94 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:09.879934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118944c0-ecdd-48c6-9be9-67885623fc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:09.899664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-118944c0-ecdd-48c6-9be9-67885623fc94 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:09.900919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-118944c0-ecdd-48c6-9be9-67885623fc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:09.909825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 692/1378] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:09.954508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3eb2-6eba-43ea-8448-0ddce316230e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:09.971015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3eb2-6eba-43ea-8448-0ddce316230e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:09.977704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e280e27-5d92-458d-8d8e-648adb7f80c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:09.978375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e280e27-5d92-458d-8d8e-648adb7f80c9 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:09.978660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e280e27-5d92-458d-8d8e-648adb7f80c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:09.978940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e280e27-5d92-458d-8d8e-648adb7f80c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:09.993533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e280e27-5d92-458d-8d8e-648adb7f80c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:09.995169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e280e27-5d92-458d-8d8e-648adb7f80c9 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:09.999307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:10.008087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 687/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:12.348449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e2a78-7879-406f-bd46-e6ea6e01b9dc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:12.348996 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e2a78-7879-406f-bd46-e6ea6e01b9dc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:12.349148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e2a78-7879-406f-bd46-e6ea6e01b9dc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:12.350614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 693/1380] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:12.356609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a50e1-797a-46a8-a7ed-201c84672346 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:12.357112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a50e1-797a-46a8-a7ed-201c84672346 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:12.357303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a50e1-797a-46a8-a7ed-201c84672346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:12.383237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-747a50e1-797a-46a8-a7ed-201c84672346 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:12.384535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-747a50e1-797a-46a8-a7ed-201c84672346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:12.406710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 688/1381] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:12.458807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e2a78-7879-406f-bd46-e6ea6e01b9dc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:12.479736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e2a78-7879-406f-bd46-e6ea6e01b9dc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:12.490708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58558c46-4be0-44e0-8e69-039fc15309e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:12.491808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58558c46-4be0-44e0-8e69-039fc15309e4 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:12.492427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58558c46-4be0-44e0-8e69-039fc15309e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:12.492808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58558c46-4be0-44e0-8e69-039fc15309e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:12.509076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58558c46-4be0-44e0-8e69-039fc15309e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:12.510682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58558c46-4be0-44e0-8e69-039fc15309e4 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:12.539121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 694/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:15.043531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a246bf-2592-4e88-b8f2-88bb232b7439 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:15.044136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a246bf-2592-4e88-b8f2-88bb232b7439 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:15.044136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a246bf-2592-4e88-b8f2-88bb232b7439 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:15.053680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834574dc-00a4-467a-abe6-344e245bfa20 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:15.053680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834574dc-00a4-467a-abe6-344e245bfa20 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:15.053680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834574dc-00a4-467a-abe6-344e245bfa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:15.055307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 689/1383] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:15.073579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-834574dc-00a4-467a-abe6-344e245bfa20 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:15.074804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-834574dc-00a4-467a-abe6-344e245bfa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:15.084943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 695/1384] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:15.131281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a246bf-2592-4e88-b8f2-88bb232b7439 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:15.148283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a246bf-2592-4e88-b8f2-88bb232b7439 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:15.154204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef3a5-b51c-4079-928f-9043e2f9ace7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:15.154876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef3a5-b51c-4079-928f-9043e2f9ace7 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:15.155158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef3a5-b51c-4079-928f-9043e2f9ace7 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:15.155433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef3a5-b51c-4079-928f-9043e2f9ace7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:15.165595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ef3a5-b51c-4079-928f-9043e2f9ace7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:15.167463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ef3a5-b51c-4079-928f-9043e2f9ace7 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:15.179345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 690/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:17.384386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9341fba-9630-469e-8e4e-01a2570626bc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:17.385207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9341fba-9630-469e-8e4e-01a2570626bc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:17.385207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9341fba-9630-469e-8e4e-01a2570626bc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:17.386248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 696/1386] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:17.393094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d459b6-7d4b-4293-8dc6-88807c2dc616 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:17.394251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d459b6-7d4b-4293-8dc6-88807c2dc616 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:17.394251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d459b6-7d4b-4293-8dc6-88807c2dc616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:17.423755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d8d459b6-7d4b-4293-8dc6-88807c2dc616 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:17.426679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8d459b6-7d4b-4293-8dc6-88807c2dc616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:17.440434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 691/1387] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:17.492255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9341fba-9630-469e-8e4e-01a2570626bc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:17.512186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9341fba-9630-469e-8e4e-01a2570626bc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:17.519546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3642b-7da3-4481-b60c-8d1542574d68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:17.520396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3642b-7da3-4481-b60c-8d1542574d68 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:17.521438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3642b-7da3-4481-b60c-8d1542574d68 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:17.521438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3642b-7da3-4481-b60c-8d1542574d68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:17.534699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3642b-7da3-4481-b60c-8d1542574d68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:17.537066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3642b-7da3-4481-b60c-8d1542574d68 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:17.552435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 697/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:21:20.108363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ac693-e7e6-4b41-a839-5a4d921702ca None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:20.108923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ac693-e7e6-4b41-a839-5a4d921702ca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:20.109188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ac693-e7e6-4b41-a839-5a4d921702ca None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:20.110702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 692/1389] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:21:20.116769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8455e-0f8c-422e-a133-bf78b6de6f1e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:20.117262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8455e-0f8c-422e-a133-bf78b6de6f1e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:20.117363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8455e-0f8c-422e-a133-bf78b6de6f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:20.137330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bfb8455e-0f8c-422e-a133-bf78b6de6f1e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:20.138530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bfb8455e-0f8c-422e-a133-bf78b6de6f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:20.147445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 698/1390] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:20.197132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ac693-e7e6-4b41-a839-5a4d921702ca None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:20.214712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ac693-e7e6-4b41-a839-5a4d921702ca None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:20.220508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db313ee2-d1da-4ed4-9e9a-b7e3040fafff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:20.222610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db313ee2-d1da-4ed4-9e9a-b7e3040fafff service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:20.222840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db313ee2-d1da-4ed4-9e9a-b7e3040fafff service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:20.223129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db313ee2-d1da-4ed4-9e9a-b7e3040fafff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:20.233042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db313ee2-d1da-4ed4-9e9a-b7e3040fafff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:20.234745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db313ee2-d1da-4ed4-9e9a-b7e3040fafff service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:20.245007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 693/1391] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:22.925809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910ccf-ef1a-4f03-bbaa-dc57419b9ba2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:22.925809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910ccf-ef1a-4f03-bbaa-dc57419b9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:22.926470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910ccf-ef1a-4f03-bbaa-dc57419b9ba2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:22.927474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 699/1392] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:22.933162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67466ee-d3ba-476b-b34f-2b98eea68c47 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:22.933638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67466ee-d3ba-476b-b34f-2b98eea68c47 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:22.933736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67466ee-d3ba-476b-b34f-2b98eea68c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:22.954607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c67466ee-d3ba-476b-b34f-2b98eea68c47 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:22.954607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c67466ee-d3ba-476b-b34f-2b98eea68c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:22.968031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 694/1393] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:23.013954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910ccf-ef1a-4f03-bbaa-dc57419b9ba2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:23.028581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910ccf-ef1a-4f03-bbaa-dc57419b9ba2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:23.035152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ffd3d-a648-49e7-8cfb-031cba85798d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:23.035817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ffd3d-a648-49e7-8cfb-031cba85798d service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:23.036132 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ffd3d-a648-49e7-8cfb-031cba85798d service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:23.036408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ffd3d-a648-49e7-8cfb-031cba85798d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:23.045701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ffd3d-a648-49e7-8cfb-031cba85798d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:23.047285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ffd3d-a648-49e7-8cfb-031cba85798d service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:23.058645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 700/1394] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:24.660253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34322ef9-218e-4f4a-a0e4-f2aade82f09a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:24.660866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34322ef9-218e-4f4a-a0e4-f2aade82f09a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:24.660866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34322ef9-218e-4f4a-a0e4-f2aade82f09a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:24.662196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 695/1395] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:24.667939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc83e7-ca7a-4bb9-b562-f683b7594570 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:24.668405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc83e7-ca7a-4bb9-b562-f683b7594570 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:24.668550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc83e7-ca7a-4bb9-b562-f683b7594570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:24.675457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=52 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:24.688927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5dfc83e7-ca7a-4bb9-b562-f683b7594570 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:24.691233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5dfc83e7-ca7a-4bb9-b562-f683b7594570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:24.699439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 701/1396] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:24.757403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cd5c3-1723-4151-bee3-19e3a6474534 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:24.757665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cd5c3-1723-4151-bee3-19e3a6474534 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:24.757951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cd5c3-1723-4151-bee3-19e3a6474534 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:24.759339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 696/1397] 10.222.0.30 () {54 vars in 962 bytes} [Thu Jun 16 18:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:24.764768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7fd95-2e3f-4cbe-ba11-bb5488f9ef7a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:24.765065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7fd95-2e3f-4cbe-ba11-bb5488f9ef7a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:24.765347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7fd95-2e3f-4cbe-ba11-bb5488f9ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:24.822234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-33d7fd95-2e3f-4cbe-ba11-bb5488f9ef7a None None] MFA Rules not processed for user `80ea2163e08f4443a3d6af03a59351f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:24.823451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-33d7fd95-2e3f-4cbe-ba11-bb5488f9ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:24.891325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 702/1398] 10.222.0.30 () {58 vars in 1059 bytes} [Thu Jun 16 18:21:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:24.899614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cd5c3-1723-4151-bee3-19e3a6474534 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:24.912005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cd5c3-1723-4151-bee3-19e3a6474534 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:24.916119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284cf49-a8c6-44d7-8241-022a6c003377 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:24.916787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284cf49-a8c6-44d7-8241-022a6c003377 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:24.917077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284cf49-a8c6-44d7-8241-022a6c003377 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:24.917361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284cf49-a8c6-44d7-8241-022a6c003377 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:24.929350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284cf49-a8c6-44d7-8241-022a6c003377 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:24.931162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284cf49-a8c6-44d7-8241-022a6c003377 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:24.942225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 697/1399] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:25.294926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aaa07-4c60-4c08-bf4e-a98779b0a8ab None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:25.295325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aaa07-4c60-4c08-bf4e-a98779b0a8ab None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:25.295498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aaa07-4c60-4c08-bf4e-a98779b0a8ab None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:25.297261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 703/1400] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:25.303272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff46a-c6c8-4dd4-8aad-70834a5355ce None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:25.303570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff46a-c6c8-4dd4-8aad-70834a5355ce None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:25.303851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff46a-c6c8-4dd4-8aad-70834a5355ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:25.323103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fdaff46a-c6c8-4dd4-8aad-70834a5355ce None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:25.324371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fdaff46a-c6c8-4dd4-8aad-70834a5355ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:25.339118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 698/1401] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:25.386111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aaa07-4c60-4c08-bf4e-a98779b0a8ab None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:25.403347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aaa07-4c60-4c08-bf4e-a98779b0a8ab None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:25.409799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbcf8-6df2-4e84-bbc9-bf768b25fb83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:25.410475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbcf8-6df2-4e84-bbc9-bf768b25fb83 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:25.410753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbcf8-6df2-4e84-bbc9-bf768b25fb83 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:25.411022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbcf8-6df2-4e84-bbc9-bf768b25fb83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:25.420675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fbcf8-6df2-4e84-bbc9-bf768b25fb83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:25.422283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fbcf8-6df2-4e84-bbc9-bf768b25fb83 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:25.433403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 704/1402] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:25.695908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3a5fc-eb84-4c42-af85-17d7dc113bb1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:25.696761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3a5fc-eb84-4c42-af85-17d7dc113bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:25.697165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3a5fc-eb84-4c42-af85-17d7dc113bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:25.759740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-51f3a5fc-eb84-4c42-af85-17d7dc113bb1 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:25.761031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-51f3a5fc-eb84-4c42-af85-17d7dc113bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:25.834016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 699/1403] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:21:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:25.879320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bd89b-daf0-46d0-8d8f-9a88cf9008ad None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:25.879320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bd89b-daf0-46d0-8d8f-9a88cf9008ad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:25.879320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bd89b-daf0-46d0-8d8f-9a88cf9008ad None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:25.879320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 705/1404] 10.222.0.30 () {54 vars in 942 bytes} [Thu Jun 16 18:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:21:25.886241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82736-8da9-44d6-afa2-6de97844add0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:25.886651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82736-8da9-44d6-afa2-6de97844add0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:25.886999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82736-8da9-44d6-afa2-6de97844add0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:25.942753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2eb82736-8da9-44d6-afa2-6de97844add0 None None] MFA Rules not processed for user `04b5081e9bf640679c72beb1b18f3e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:25.943888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2eb82736-8da9-44d6-afa2-6de97844add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:25.980466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:26.021263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 700/1405] 10.222.0.30 () {58 vars in 1039 bytes} [Thu Jun 16 18:21:25 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:26.030661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bd89b-daf0-46d0-8d8f-9a88cf9008ad None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:26.046756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bd89b-daf0-46d0-8d8f-9a88cf9008ad None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:26.051945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2baa07-eb3e-4293-9287-a179807ebcd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:26.052884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2baa07-eb3e-4293-9287-a179807ebcd3 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:26.053250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2baa07-eb3e-4293-9287-a179807ebcd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:26.053601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2baa07-eb3e-4293-9287-a179807ebcd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:26.062314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2baa07-eb3e-4293-9287-a179807ebcd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:26.064071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2baa07-eb3e-4293-9287-a179807ebcd3 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:26.072963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 706/1406] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:21:27.694356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d738-886c-4550-b07e-2fb11263f279 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:27.694982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d738-886c-4550-b07e-2fb11263f279 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:27.694982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d738-886c-4550-b07e-2fb11263f279 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:27.700111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 701/1407] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:27.702906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24ca15-d912-4ca9-8799-6b0f6365c62a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:27.703391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24ca15-d912-4ca9-8799-6b0f6365c62a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:27.703482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24ca15-d912-4ca9-8799-6b0f6365c62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:27.725267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ba24ca15-d912-4ca9-8799-6b0f6365c62a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:27.726496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ba24ca15-d912-4ca9-8799-6b0f6365c62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:27.734530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 707/1408] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:27.785455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15d738-886c-4550-b07e-2fb11263f279 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:27.806324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15d738-886c-4550-b07e-2fb11263f279 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:27.812119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c41f9-f7bf-4119-91c8-a6b5ad76216f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:27.812924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c41f9-f7bf-4119-91c8-a6b5ad76216f service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:27.813211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c41f9-f7bf-4119-91c8-a6b5ad76216f service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:27.813485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c41f9-f7bf-4119-91c8-a6b5ad76216f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:27.823299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c41f9-f7bf-4119-91c8-a6b5ad76216f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:27.824931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c41f9-f7bf-4119-91c8-a6b5ad76216f service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:27.840907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 702/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:37.718042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=55 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:37.831856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=21 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:49.448015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006507e3-6be5-4a7a-a6a4-70f8384b9fb6 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:49.448626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006507e3-6be5-4a7a-a6a4-70f8384b9fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:49.448626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006507e3-6be5-4a7a-a6a4-70f8384b9fb6 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:49.450014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 708/1410] 10.222.0.30 () {54 vars in 918 bytes} [Thu Jun 16 18:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:49.455480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984097b-1c7a-46d8-b3e2-9914eaa388c0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:49.455935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984097b-1c7a-46d8-b3e2-9914eaa388c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:49.456075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984097b-1c7a-46d8-b3e2-9914eaa388c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:49.475146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9984097b-1c7a-46d8-b3e2-9914eaa388c0 None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:49.476414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9984097b-1c7a-46d8-b3e2-9914eaa388c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:49.485067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 703/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Jun 16 18:21:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:49.499783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e30392-160d-46c2-93c3-bc373c977caa None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:49.500098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e30392-160d-46c2-93c3-bc373c977caa None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:49.500381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e30392-160d-46c2-93c3-bc373c977caa None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:49.502852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 709/1412] 10.222.0.30 () {54 vars in 918 bytes} [Thu Jun 16 18:21:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:49.507799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e8cf1-3cd3-47eb-9645-040047f9c70d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:49.508123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e8cf1-3cd3-47eb-9645-040047f9c70d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:49.508410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e8cf1-3cd3-47eb-9645-040047f9c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:49.527860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b79e8cf1-3cd3-47eb-9645-040047f9c70d None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:49.529098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b79e8cf1-3cd3-47eb-9645-040047f9c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:49.538816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 704/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Jun 16 18:21:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:52.141293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb183c56-bca5-4ea3-9c2b-7e315073ddfc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:52.142501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb183c56-bca5-4ea3-9c2b-7e315073ddfc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:52.142935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb183c56-bca5-4ea3-9c2b-7e315073ddfc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:52.144358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:21:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:52.149554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a531-afb8-4d5e-8811-f24c7295c3b2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:52.149963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a531-afb8-4d5e-8811-f24c7295c3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:52.150134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a531-afb8-4d5e-8811-f24c7295c3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:52.169033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fb75a531-afb8-4d5e-8811-f24c7295c3b2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:52.170226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fb75a531-afb8-4d5e-8811-f24c7295c3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:52.178240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:52.222533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb183c56-bca5-4ea3-9c2b-7e315073ddfc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:52.237653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb183c56-bca5-4ea3-9c2b-7e315073ddfc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:52.243019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f71ee-9401-495f-b432-540abdc0b786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:52.243661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f71ee-9401-495f-b432-540abdc0b786 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:52.243938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f71ee-9401-495f-b432-540abdc0b786 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:52.246152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f71ee-9401-495f-b432-540abdc0b786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:52.254107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f71ee-9401-495f-b432-540abdc0b786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:52.255735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f71ee-9401-495f-b432-540abdc0b786 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:52.265282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 711/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:53.226695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedfc9a-4ffc-43ad-b3dc-1f98507decd8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:53.227248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedfc9a-4ffc-43ad-b3dc-1f98507decd8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:53.227248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedfc9a-4ffc-43ad-b3dc-1f98507decd8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:53.228661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:53.233854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209fd0-5caf-4e70-ac24-5e4f8ce9ed10 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:53.234250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209fd0-5caf-4e70-ac24-5e4f8ce9ed10 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:53.234408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209fd0-5caf-4e70-ac24-5e4f8ce9ed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:53.252852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-60209fd0-5caf-4e70-ac24-5e4f8ce9ed10 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:53.254040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-60209fd0-5caf-4e70-ac24-5e4f8ce9ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:53.261895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:53.419212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedfc9a-4ffc-43ad-b3dc-1f98507decd8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:53.434282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedfc9a-4ffc-43ad-b3dc-1f98507decd8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:53.440326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0e65d-7c67-4eba-8c09-621e9dd199a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:53.440976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e65d-7c67-4eba-8c09-621e9dd199a1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:53.441262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e65d-7c67-4eba-8c09-621e9dd199a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:53.441539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e65d-7c67-4eba-8c09-621e9dd199a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:53.449834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0e65d-7c67-4eba-8c09-621e9dd199a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:53.451416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0e65d-7c67-4eba-8c09-621e9dd199a1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:53.462013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 707/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:54.413084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844d0c4-b234-42c3-a12a-fe1f363812f3 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:54.413679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844d0c4-b234-42c3-a12a-fe1f363812f3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:54.413679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844d0c4-b234-42c3-a12a-fe1f363812f3 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:54.415020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:54.420320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afcf3c-c9f0-4124-b91e-fde0e7f34572 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:54.420761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afcf3c-c9f0-4124-b91e-fde0e7f34572 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:54.420876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afcf3c-c9f0-4124-b91e-fde0e7f34572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:54.440378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-05afcf3c-c9f0-4124-b91e-fde0e7f34572 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:54.441589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-05afcf3c-c9f0-4124-b91e-fde0e7f34572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:54.449677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:54.604513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844d0c4-b234-42c3-a12a-fe1f363812f3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:54.619717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844d0c4-b234-42c3-a12a-fe1f363812f3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:54.625358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc54bdf-6e93-4ecf-af57-0ddf53d7ba44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:54.627080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc54bdf-6e93-4ecf-af57-0ddf53d7ba44 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:54.627399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc54bdf-6e93-4ecf-af57-0ddf53d7ba44 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:54.627654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc54bdf-6e93-4ecf-af57-0ddf53d7ba44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:54.636563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc54bdf-6e93-4ecf-af57-0ddf53d7ba44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:54.638136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc54bdf-6e93-4ecf-af57-0ddf53d7ba44 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:54.648455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 714/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:55.568655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa02fb-2b64-4de3-b28c-acfd95a4f7bb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:55.569230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa02fb-2b64-4de3-b28c-acfd95a4f7bb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:55.569230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa02fb-2b64-4de3-b28c-acfd95a4f7bb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:55.570606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:55.575730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8385-4026-4721-82c4-0e97f3a6b1b9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:55.576174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8385-4026-4721-82c4-0e97f3a6b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:55.576329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8385-4026-4721-82c4-0e97f3a6b1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:55.595596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-211e8385-4026-4721-82c4-0e97f3a6b1b9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:55.597222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-211e8385-4026-4721-82c4-0e97f3a6b1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:55.605729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:55.762019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa02fb-2b64-4de3-b28c-acfd95a4f7bb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:55.776973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa02fb-2b64-4de3-b28c-acfd95a4f7bb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:55.783483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2c0ce-dc1e-46ce-8d28-ae0841984e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:55.784173 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2c0ce-dc1e-46ce-8d28-ae0841984e8f service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:55.784459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2c0ce-dc1e-46ce-8d28-ae0841984e8f service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:55.784738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2c0ce-dc1e-46ce-8d28-ae0841984e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:55.793579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2c0ce-dc1e-46ce-8d28-ae0841984e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:55.795182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2c0ce-dc1e-46ce-8d28-ae0841984e8f service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:55.806632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 710/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:56.735753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284c4e-a7f7-479c-82d3-14d4d7fd340d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:56.736365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284c4e-a7f7-479c-82d3-14d4d7fd340d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:56.736365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284c4e-a7f7-479c-82d3-14d4d7fd340d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:56.737729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:56.742844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2259a7-269c-4870-9abf-918f3a096e82 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:56.743265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2259a7-269c-4870-9abf-918f3a096e82 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:56.743409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2259a7-269c-4870-9abf-918f3a096e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:56.762741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1b2259a7-269c-4870-9abf-918f3a096e82 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:56.763911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1b2259a7-269c-4870-9abf-918f3a096e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:56.773305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:56.926641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284c4e-a7f7-479c-82d3-14d4d7fd340d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:56.941331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284c4e-a7f7-479c-82d3-14d4d7fd340d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:56.946739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffec2aa-33e1-4376-9a1e-be469b0acc72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:56.947378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffec2aa-33e1-4376-9a1e-be469b0acc72 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:56.947660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffec2aa-33e1-4376-9a1e-be469b0acc72 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:56.947937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffec2aa-33e1-4376-9a1e-be469b0acc72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:56.957416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffec2aa-33e1-4376-9a1e-be469b0acc72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:56.959002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffec2aa-33e1-4376-9a1e-be469b0acc72 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:56.968770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 717/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:57.836657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908fa33-3245-467e-91d4-e39e6eaa6d2c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:57.837335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908fa33-3245-467e-91d4-e39e6eaa6d2c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:57.837335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908fa33-3245-467e-91d4-e39e6eaa6d2c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:57.838575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:57.843760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc072fb-ad45-4f75-9b81-d70ceae93492 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:57.844327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc072fb-ad45-4f75-9b81-d70ceae93492 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:57.844478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc072fb-ad45-4f75-9b81-d70ceae93492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:57.862296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0bc072fb-ad45-4f75-9b81-d70ceae93492 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:57.863524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0bc072fb-ad45-4f75-9b81-d70ceae93492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:57.871783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:21:58.026164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908fa33-3245-467e-91d4-e39e6eaa6d2c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:58.041711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908fa33-3245-467e-91d4-e39e6eaa6d2c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:58.047429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f19e9-bcf1-485c-a458-fe20c40bd6e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:58.048280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f19e9-bcf1-485c-a458-fe20c40bd6e1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:58.048567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f19e9-bcf1-485c-a458-fe20c40bd6e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:58.048840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f19e9-bcf1-485c-a458-fe20c40bd6e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:58.057787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f19e9-bcf1-485c-a458-fe20c40bd6e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:58.059365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f19e9-bcf1-485c-a458-fe20c40bd6e1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:58.069717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 713/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:58.916808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291319d8-f000-445e-9a60-55fc36a9df92 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:58.917384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291319d8-f000-445e-9a60-55fc36a9df92 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:58.917384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291319d8-f000-445e-9a60-55fc36a9df92 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:58.918788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:21:58.924579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f46bb-5263-4cde-bee6-5cfddb608e47 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:58.925040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f46bb-5263-4cde-bee6-5cfddb608e47 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:58.925165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f46bb-5263-4cde-bee6-5cfddb608e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:58.944003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e54f46bb-5263-4cde-bee6-5cfddb608e47 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:21:58.945239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e54f46bb-5263-4cde-bee6-5cfddb608e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:58.953613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:21:59.108483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291319d8-f000-445e-9a60-55fc36a9df92 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:59.124942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291319d8-f000-445e-9a60-55fc36a9df92 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:59.131033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d954c-1f88-46d2-9109-ca0fce85aab9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:59.131677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d954c-1f88-46d2-9109-ca0fce85aab9 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:59.131952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d954c-1f88-46d2-9109-ca0fce85aab9 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:59.132266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d954c-1f88-46d2-9109-ca0fce85aab9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:59.140675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d954c-1f88-46d2-9109-ca0fce85aab9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:59.142461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d954c-1f88-46d2-9109-ca0fce85aab9 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:59.151773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 720/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:00.029986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8bd27-2afc-4dc5-b00e-4454d8992c13 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:00.030568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8bd27-2afc-4dc5-b00e-4454d8992c13 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:00.030568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8bd27-2afc-4dc5-b00e-4454d8992c13 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:00.031935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:00.037085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab15825-f166-4494-8673-9dbf218e4301 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:00.037788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab15825-f166-4494-8673-9dbf218e4301 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:00.038055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab15825-f166-4494-8673-9dbf218e4301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:00.056635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1ab15825-f166-4494-8673-9dbf218e4301 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:00.057845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ab15825-f166-4494-8673-9dbf218e4301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:00.066321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:00.218189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8bd27-2afc-4dc5-b00e-4454d8992c13 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:00.233098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8bd27-2afc-4dc5-b00e-4454d8992c13 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:00.239362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905814-84f9-4f2e-85ee-ad45e12f21ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:00.240014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905814-84f9-4f2e-85ee-ad45e12f21ff service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:00.240328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905814-84f9-4f2e-85ee-ad45e12f21ff service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:00.240607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905814-84f9-4f2e-85ee-ad45e12f21ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:00.248895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68905814-84f9-4f2e-85ee-ad45e12f21ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:00.250479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68905814-84f9-4f2e-85ee-ad45e12f21ff service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:00.260532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 716/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:01.139058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c085a7-d620-4976-8fbe-dfa7a238b05f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:01.139593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c085a7-d620-4976-8fbe-dfa7a238b05f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:01.139593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c085a7-d620-4976-8fbe-dfa7a238b05f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:01.140945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:01.146568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b0bf-1c42-49dd-8eab-56e2c9dd9441 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:01.146991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b0bf-1c42-49dd-8eab-56e2c9dd9441 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:01.147097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b0bf-1c42-49dd-8eab-56e2c9dd9441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:01.165936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fe91b0bf-1c42-49dd-8eab-56e2c9dd9441 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:01.167130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fe91b0bf-1c42-49dd-8eab-56e2c9dd9441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:01.175311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:01.326354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c085a7-d620-4976-8fbe-dfa7a238b05f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:01.342145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c085a7-d620-4976-8fbe-dfa7a238b05f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:01.348773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ab8b0-9f12-4fc0-ab94-88dcbd516d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:01.349428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ab8b0-9f12-4fc0-ab94-88dcbd516d76 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:01.349736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ab8b0-9f12-4fc0-ab94-88dcbd516d76 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:01.349994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ab8b0-9f12-4fc0-ab94-88dcbd516d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:01.358931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ab8b0-9f12-4fc0-ab94-88dcbd516d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:01.360547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ab8b0-9f12-4fc0-ab94-88dcbd516d76 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:01.369878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 723/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:03.453416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab94c-9c7e-45d6-9f22-270faccc282a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:03.453984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab94c-9c7e-45d6-9f22-270faccc282a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:03.453984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab94c-9c7e-45d6-9f22-270faccc282a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:03.455398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:03.460900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154d3d-0f8e-4f84-afb3-58a14cc18a18 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:03.461340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154d3d-0f8e-4f84-afb3-58a14cc18a18 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:03.461488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154d3d-0f8e-4f84-afb3-58a14cc18a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:03.478947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bd154d3d-0f8e-4f84-afb3-58a14cc18a18 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:03.480204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bd154d3d-0f8e-4f84-afb3-58a14cc18a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:03.488685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:03.527141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfab94c-9c7e-45d6-9f22-270faccc282a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:03.538266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfab94c-9c7e-45d6-9f22-270faccc282a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:03.542148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faae647-52d7-4048-b980-b44bb14d00bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:03.542789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faae647-52d7-4048-b980-b44bb14d00bc service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:03.543068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faae647-52d7-4048-b980-b44bb14d00bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:03.543340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faae647-52d7-4048-b980-b44bb14d00bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:03.552924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faae647-52d7-4048-b980-b44bb14d00bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:03.554359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faae647-52d7-4048-b980-b44bb14d00bc service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:03.564612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 719/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:05.872470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d189a9-0a88-4c1a-aabf-11900b760334 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:05.873051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d189a9-0a88-4c1a-aabf-11900b760334 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:05.873051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d189a9-0a88-4c1a-aabf-11900b760334 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:05.874376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:05.879980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b596d-b833-4f5f-8932-a209418f92c4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:05.880465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b596d-b833-4f5f-8932-a209418f92c4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:05.880573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b596d-b833-4f5f-8932-a209418f92c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:05.899580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ee4b596d-b833-4f5f-8932-a209418f92c4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:05.900837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ee4b596d-b833-4f5f-8932-a209418f92c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:05.909904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:05.949458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d189a9-0a88-4c1a-aabf-11900b760334 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:05.960835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d189a9-0a88-4c1a-aabf-11900b760334 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:05.964311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bfd0a-1c56-4798-a80a-bffb00512480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:05.964949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bfd0a-1c56-4798-a80a-bffb00512480 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:05.965224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bfd0a-1c56-4798-a80a-bffb00512480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:05.965491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bfd0a-1c56-4798-a80a-bffb00512480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:05.977537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bfd0a-1c56-4798-a80a-bffb00512480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:05.978970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bfd0a-1c56-4798-a80a-bffb00512480 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:05.989197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 726/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:08.451106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3973b29-b2b5-46b5-b503-f507f52ae967 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:08.451701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3973b29-b2b5-46b5-b503-f507f52ae967 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:08.451701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3973b29-b2b5-46b5-b503-f507f52ae967 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:08.453071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:08.459030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b057f7-7e51-473f-a65a-dc0f96202259 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:08.459448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b057f7-7e51-473f-a65a-dc0f96202259 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:08.459586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b057f7-7e51-473f-a65a-dc0f96202259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:08.477211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-27b057f7-7e51-473f-a65a-dc0f96202259 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:08.478427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-27b057f7-7e51-473f-a65a-dc0f96202259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:08.486672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:08.526479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3973b29-b2b5-46b5-b503-f507f52ae967 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:08.537644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3973b29-b2b5-46b5-b503-f507f52ae967 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:08.541420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6312e-aa22-4604-a99b-0676201ff186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:08.542060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6312e-aa22-4604-a99b-0676201ff186 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:08.542368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6312e-aa22-4604-a99b-0676201ff186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:08.542645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6312e-aa22-4604-a99b-0676201ff186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:08.552415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6312e-aa22-4604-a99b-0676201ff186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:08.553848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6312e-aa22-4604-a99b-0676201ff186 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:08.564462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 722/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:10.874606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977429ab-533b-4e29-9ffc-05a732c82895 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:10.875094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977429ab-533b-4e29-9ffc-05a732c82895 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:10.875859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977429ab-533b-4e29-9ffc-05a732c82895 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:10.877518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:10.883015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03a70-9820-4d04-9cc0-537460bcecc2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:10.883417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03a70-9820-4d04-9cc0-537460bcecc2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:10.883576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03a70-9820-4d04-9cc0-537460bcecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:10.901816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3ef03a70-9820-4d04-9cc0-537460bcecc2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:10.903024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3ef03a70-9820-4d04-9cc0-537460bcecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:10.911047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:10.950555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977429ab-533b-4e29-9ffc-05a732c82895 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:10.961254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977429ab-533b-4e29-9ffc-05a732c82895 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:10.964730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf576-2074-4cbb-8411-dd9d4236ecb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:10.965359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf576-2074-4cbb-8411-dd9d4236ecb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:10.965636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf576-2074-4cbb-8411-dd9d4236ecb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:10.965903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf576-2074-4cbb-8411-dd9d4236ecb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:10.975558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf576-2074-4cbb-8411-dd9d4236ecb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:10.977069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf576-2074-4cbb-8411-dd9d4236ecb8 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:10.987296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 729/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:13.218060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7e68b-be9a-43d7-8803-5aa3b8f33b7e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:13.218603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7e68b-be9a-43d7-8803-5aa3b8f33b7e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:13.218603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7e68b-be9a-43d7-8803-5aa3b8f33b7e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:13.220311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:13.225425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6963d0-6fce-441f-9d31-6fd4ea530988 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:13.225870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6963d0-6fce-441f-9d31-6fd4ea530988 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:13.226015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6963d0-6fce-441f-9d31-6fd4ea530988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:13.244761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0c6963d0-6fce-441f-9d31-6fd4ea530988 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:13.245963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0c6963d0-6fce-441f-9d31-6fd4ea530988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:13.255106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:13.292742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7e68b-be9a-43d7-8803-5aa3b8f33b7e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:13.304365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7e68b-be9a-43d7-8803-5aa3b8f33b7e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:13.307878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39fd6-3b8f-421a-b1d5-298e8e299128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:13.308573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39fd6-3b8f-421a-b1d5-298e8e299128 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:13.308856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39fd6-3b8f-421a-b1d5-298e8e299128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:13.309130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39fd6-3b8f-421a-b1d5-298e8e299128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:13.318746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f39fd6-3b8f-421a-b1d5-298e8e299128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:13.320198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f39fd6-3b8f-421a-b1d5-298e8e299128 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:13.329632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 725/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:15.594292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c83fb-15e9-4c5f-a128-545123079cb0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:15.594831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c83fb-15e9-4c5f-a128-545123079cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:15.595007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c83fb-15e9-4c5f-a128-545123079cb0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:15.596434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:15.601912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40104ed-5246-424d-9ca0-a9d9c078c57c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:15.602312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40104ed-5246-424d-9ca0-a9d9c078c57c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:15.602471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40104ed-5246-424d-9ca0-a9d9c078c57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:15.621231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c40104ed-5246-424d-9ca0-a9d9c078c57c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:15.622443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c40104ed-5246-424d-9ca0-a9d9c078c57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:15.631279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:15.669294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c83fb-15e9-4c5f-a128-545123079cb0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:15.680899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c83fb-15e9-4c5f-a128-545123079cb0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:15.684593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46423a62-85fd-4853-a829-fb57b8f5ac27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:15.685251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46423a62-85fd-4853-a829-fb57b8f5ac27 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:15.685533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46423a62-85fd-4853-a829-fb57b8f5ac27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:15.685807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46423a62-85fd-4853-a829-fb57b8f5ac27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:15.698345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46423a62-85fd-4853-a829-fb57b8f5ac27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:15.699810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46423a62-85fd-4853-a829-fb57b8f5ac27 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:15.710156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 732/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:17.949941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f502d-5d30-449f-8581-0dcffee67c51 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:17.950520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f502d-5d30-449f-8581-0dcffee67c51 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:17.950520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f502d-5d30-449f-8581-0dcffee67c51 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:17.951872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:17.958468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03cfef-178d-4ef5-8f0c-28b648d75397 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:17.958939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03cfef-178d-4ef5-8f0c-28b648d75397 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:17.959050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03cfef-178d-4ef5-8f0c-28b648d75397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:17.965161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=70 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:17.979597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ce03cfef-178d-4ef5-8f0c-28b648d75397 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:17.980841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ce03cfef-178d-4ef5-8f0c-28b648d75397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:17.988953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:18.027490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f502d-5d30-449f-8581-0dcffee67c51 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:18.032974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=73 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:18.039601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f502d-5d30-449f-8581-0dcffee67c51 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:18.043395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe0373-4dd4-445c-9fec-893203e49bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:18.044065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0373-4dd4-445c-9fec-893203e49bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:18.044353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0373-4dd4-445c-9fec-893203e49bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:18.044636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0373-4dd4-445c-9fec-893203e49bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:18.054591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe0373-4dd4-445c-9fec-893203e49bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:18.056017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe0373-4dd4-445c-9fec-893203e49bd8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:18.066031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 728/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:20.242552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d1deb-b1b7-4e9a-802e-f5f8eac785bb None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:20.243130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d1deb-b1b7-4e9a-802e-f5f8eac785bb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:20.243130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d1deb-b1b7-4e9a-802e-f5f8eac785bb None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:20.244564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:20.250640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280c76-140f-478f-a1f1-eadbb01bd15b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:20.251449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280c76-140f-478f-a1f1-eadbb01bd15b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:20.251732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4280c76-140f-478f-a1f1-eadbb01bd15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:20.274410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c4280c76-140f-478f-a1f1-eadbb01bd15b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:20.275643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c4280c76-140f-478f-a1f1-eadbb01bd15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:20.283905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:20.323414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d1deb-b1b7-4e9a-802e-f5f8eac785bb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:20.325206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a81d1deb-b1b7-4e9a-802e-f5f8eac785bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:20.339933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d1deb-b1b7-4e9a-802e-f5f8eac785bb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:20.343521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265625be-9a4f-4c62-82a1-63c694b9c205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:20.344749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265625be-9a4f-4c62-82a1-63c694b9c205 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:20.345030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265625be-9a4f-4c62-82a1-63c694b9c205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:20.345302 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265625be-9a4f-4c62-82a1-63c694b9c205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:20.354423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265625be-9a4f-4c62-82a1-63c694b9c205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:20.355847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265625be-9a4f-4c62-82a1-63c694b9c205 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:20.365716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 735/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:22.576965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623ea9f-a9cc-47ef-a18c-527c22de82ae None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:22.577537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623ea9f-a9cc-47ef-a18c-527c22de82ae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:22.577537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623ea9f-a9cc-47ef-a18c-527c22de82ae None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:22.579573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:22.585202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93bec8-2878-489c-9574-b7bab5ed79bb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:22.585700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93bec8-2878-489c-9574-b7bab5ed79bb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:22.585798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93bec8-2878-489c-9574-b7bab5ed79bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:22.603117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5e93bec8-2878-489c-9574-b7bab5ed79bb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:22.604330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5e93bec8-2878-489c-9574-b7bab5ed79bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:22.612601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:22.650130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623ea9f-a9cc-47ef-a18c-527c22de82ae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:22.661775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623ea9f-a9cc-47ef-a18c-527c22de82ae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:22.665535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f4740-e5bd-4fa9-b96a-85f0f509eb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:22.666176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f4740-e5bd-4fa9-b96a-85f0f509eb44 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:22.666454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f4740-e5bd-4fa9-b96a-85f0f509eb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:22.666730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f4740-e5bd-4fa9-b96a-85f0f509eb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:22.676287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f4740-e5bd-4fa9-b96a-85f0f509eb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:22.677711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f4740-e5bd-4fa9-b96a-85f0f509eb44 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:22.687124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 731/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:24.895035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894ee46-0c99-4d9d-a183-ea9e7fe8b3b0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:24.895613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894ee46-0c99-4d9d-a183-ea9e7fe8b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:24.895613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894ee46-0c99-4d9d-a183-ea9e7fe8b3b0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:24.896999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:24.902515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecff61-d7cf-4864-b1aa-c26865a468f6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:24.902936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecff61-d7cf-4864-b1aa-c26865a468f6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:24.903072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecff61-d7cf-4864-b1aa-c26865a468f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:24.921671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-61ecff61-d7cf-4864-b1aa-c26865a468f6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:24.923540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-61ecff61-d7cf-4864-b1aa-c26865a468f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:24.931422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:24.967865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894ee46-0c99-4d9d-a183-ea9e7fe8b3b0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:24.979515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894ee46-0c99-4d9d-a183-ea9e7fe8b3b0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:24.983367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a22a46-a660-408a-881b-ad38ce6e3b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:24.984021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a22a46-a660-408a-881b-ad38ce6e3b84 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:24.984861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a22a46-a660-408a-881b-ad38ce6e3b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:24.985119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a22a46-a660-408a-881b-ad38ce6e3b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:24.994765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a22a46-a660-408a-881b-ad38ce6e3b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:24.996230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a22a46-a660-408a-881b-ad38ce6e3b84 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:25.005801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 738/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:34.916443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=17 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:35.001605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=17 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:45.505213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baf0a4-5c3e-49e9-a71f-69e6bb0dcad5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:45.505778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baf0a4-5c3e-49e9-a71f-69e6bb0dcad5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:45.505778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baf0a4-5c3e-49e9-a71f-69e6bb0dcad5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:45.507136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 733/1471] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:45.512740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894e05b-7f5e-48f4-ad24-18c9aeac1341 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:45.513208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894e05b-7f5e-48f4-ad24-18c9aeac1341 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:45.513441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894e05b-7f5e-48f4-ad24-18c9aeac1341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:45.532530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f894e05b-7f5e-48f4-ad24-18c9aeac1341 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:45.533742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f894e05b-7f5e-48f4-ad24-18c9aeac1341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:45.541927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 739/1472] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:45.589462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd2a3-d5f8-484d-ba71-572c2efb8706 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:45.589760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd2a3-d5f8-484d-ba71-572c2efb8706 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:45.590287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd2a3-d5f8-484d-ba71-572c2efb8706 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:45.591777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 734/1473] 10.222.0.30 () {54 vars in 962 bytes} [Thu Jun 16 18:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:45.597357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6db3-387f-4393-ae6d-804c65117ab4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:45.597668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6db3-387f-4393-ae6d-804c65117ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:45.598081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d6db3-387f-4393-ae6d-804c65117ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:45.616770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ec4d6db3-387f-4393-ae6d-804c65117ab4 None None] MFA Rules not processed for user `80ea2163e08f4443a3d6af03a59351f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:45.617967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ec4d6db3-387f-4393-ae6d-804c65117ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:45.624796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 740/1474] 10.222.0.30 () {58 vars in 1059 bytes} [Thu Jun 16 18:22:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:45.632350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd2a3-d5f8-484d-ba71-572c2efb8706 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:45.643558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd2a3-d5f8-484d-ba71-572c2efb8706 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:45.647077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb57ec-c9b6-4ffe-8d06-922508e0f32a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:45.647727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb57ec-c9b6-4ffe-8d06-922508e0f32a service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:45.648012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb57ec-c9b6-4ffe-8d06-922508e0f32a service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:45.648322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb57ec-c9b6-4ffe-8d06-922508e0f32a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:45.658551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb57ec-c9b6-4ffe-8d06-922508e0f32a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:45.660006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb57ec-c9b6-4ffe-8d06-922508e0f32a service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:45.669988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 735/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:47.983144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551db3d-19ac-44af-baa8-ea6003664b01 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:47.984470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551db3d-19ac-44af-baa8-ea6003664b01 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:47.984470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551db3d-19ac-44af-baa8-ea6003664b01 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:47.985709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 741/1476] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:47.990929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c850cc-15e9-461f-ab58-de61dcd1c865 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:47.991375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c850cc-15e9-461f-ab58-de61dcd1c865 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:47.991544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c850cc-15e9-461f-ab58-de61dcd1c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:48.010303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-24c850cc-15e9-461f-ab58-de61dcd1c865 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:48.012238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-24c850cc-15e9-461f-ab58-de61dcd1c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:48.023487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 736/1477] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:48.047878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551db3d-19ac-44af-baa8-ea6003664b01 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:48.060585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551db3d-19ac-44af-baa8-ea6003664b01 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:48.065244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:48.065894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:48.066185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:48.066462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:48.067467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:48.069270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:22:48.069270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:22:48.069270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:22:48.070741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103447-c783-414b-a3bf-d822eed480e3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:48.090546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 742/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Jun 16 18:22:48 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:22:50.333245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b5386-04f1-4c89-89db-1a295ebad5be None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:50.333801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b5386-04f1-4c89-89db-1a295ebad5be None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:50.333801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b5386-04f1-4c89-89db-1a295ebad5be None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:50.335106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 737/1479] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:22:50.340398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fd7c0-48c8-475c-9ac3-e6d718f66b24 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:50.340782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fd7c0-48c8-475c-9ac3-e6d718f66b24 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:50.340961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fd7c0-48c8-475c-9ac3-e6d718f66b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:50.360464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ae4fd7c0-48c8-475c-9ac3-e6d718f66b24 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:50.361673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ae4fd7c0-48c8-475c-9ac3-e6d718f66b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:50.369614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 743/1480] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:50.414036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b5386-04f1-4c89-89db-1a295ebad5be None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:50.430735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b5386-04f1-4c89-89db-1a295ebad5be None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:50.437169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656a65d-808d-4a50-99ad-165dc8230d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:50.437815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656a65d-808d-4a50-99ad-165dc8230d44 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:50.438091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656a65d-808d-4a50-99ad-165dc8230d44 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:50.438358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656a65d-808d-4a50-99ad-165dc8230d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:50.446673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656a65d-808d-4a50-99ad-165dc8230d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:50.448272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656a65d-808d-4a50-99ad-165dc8230d44 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:50.458049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 738/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:52.779880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c411657-0e75-4f4c-b003-68eb6d6fd27a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:52.781033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c411657-0e75-4f4c-b003-68eb6d6fd27a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:52.781662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c411657-0e75-4f4c-b003-68eb6d6fd27a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:52.783059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 744/1482] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:22:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:22:52.789055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955aa46-f61f-4bd6-bca3-9d82f9992911 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:52.789499 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955aa46-f61f-4bd6-bca3-9d82f9992911 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:52.789614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955aa46-f61f-4bd6-bca3-9d82f9992911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:52.809370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3955aa46-f61f-4bd6-bca3-9d82f9992911 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:52.810564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3955aa46-f61f-4bd6-bca3-9d82f9992911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:52.818688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 739/1483] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:52.855701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c411657-0e75-4f4c-b003-68eb6d6fd27a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:52.866740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c411657-0e75-4f4c-b003-68eb6d6fd27a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:52.870276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360ce7c-b21d-4dfb-8227-a772d684669b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:52.870913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360ce7c-b21d-4dfb-8227-a772d684669b service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:52.871367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360ce7c-b21d-4dfb-8227-a772d684669b service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:52.871643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360ce7c-b21d-4dfb-8227-a772d684669b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:52.881985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360ce7c-b21d-4dfb-8227-a772d684669b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:52.883564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360ce7c-b21d-4dfb-8227-a772d684669b service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:52.894664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 745/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:55.132705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f412002-f755-469c-ae20-6d9dbe363810 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:55.133229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f412002-f755-469c-ae20-6d9dbe363810 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:55.133229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f412002-f755-469c-ae20-6d9dbe363810 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:55.134564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 740/1485] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:55.140148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8a253-59c6-4331-b661-4df74c3ee6d1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:55.140756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8a253-59c6-4331-b661-4df74c3ee6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:55.140756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8a253-59c6-4331-b661-4df74c3ee6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:55.159291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-17c8a253-59c6-4331-b661-4df74c3ee6d1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:55.160556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-17c8a253-59c6-4331-b661-4df74c3ee6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:55.168944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 746/1486] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:55.208595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f412002-f755-469c-ae20-6d9dbe363810 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:55.219939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f412002-f755-469c-ae20-6d9dbe363810 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:55.223427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c3d98-70ea-4711-853c-7c7d56c15a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:55.224102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c3d98-70ea-4711-853c-7c7d56c15a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:55.224385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c3d98-70ea-4711-853c-7c7d56c15a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:55.224659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c3d98-70ea-4711-853c-7c7d56c15a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:55.235341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c3d98-70ea-4711-853c-7c7d56c15a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:55.236804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c3d98-70ea-4711-853c-7c7d56c15a6c service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:55.246752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 741/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:57.607126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b4d4a-27ef-4e7f-aee0-40f27195991a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:57.607694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b4d4a-27ef-4e7f-aee0-40f27195991a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:57.607694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b4d4a-27ef-4e7f-aee0-40f27195991a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:57.609215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 747/1488] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:22:57.614517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f974a3-7f94-46fe-be6a-ea28e0dd9a79 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:57.614939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f974a3-7f94-46fe-be6a-ea28e0dd9a79 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:57.615095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f974a3-7f94-46fe-be6a-ea28e0dd9a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:57.633940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e8f974a3-7f94-46fe-be6a-ea28e0dd9a79 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:57.635187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e8f974a3-7f94-46fe-be6a-ea28e0dd9a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:57.644292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 742/1489] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:57.715590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b4d4a-27ef-4e7f-aee0-40f27195991a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:57.728454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b4d4a-27ef-4e7f-aee0-40f27195991a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:57.731998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f014bc-f749-4614-896c-35fb2c315e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:57.732681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f014bc-f749-4614-896c-35fb2c315e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:57.733491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f014bc-f749-4614-896c-35fb2c315e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:57.733934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f014bc-f749-4614-896c-35fb2c315e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:57.744390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f014bc-f749-4614-896c-35fb2c315e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:57.745836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f014bc-f749-4614-896c-35fb2c315e0a service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:57.756658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 748/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:23:00.142542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47ea4-819b-4e71-a69f-964b9edc0665 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:00.143158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47ea4-819b-4e71-a69f-964b9edc0665 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:00.143158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47ea4-819b-4e71-a69f-964b9edc0665 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:00.144511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 743/1491] 10.222.0.30 () {54 vars in 926 bytes} [Thu Jun 16 18:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:23:00.149742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5624c8-4b08-43c6-b697-9376e57a1f58 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:00.150237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5624c8-4b08-43c6-b697-9376e57a1f58 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:00.150360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5624c8-4b08-43c6-b697-9376e57a1f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:00.168950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-db5624c8-4b08-43c6-b697-9376e57a1f58 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:23:00.170159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-db5624c8-4b08-43c6-b697-9376e57a1f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:00.178239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 749/1492] 10.222.0.30 () {58 vars in 1023 bytes} [Thu Jun 16 18:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:23:00.216540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a47ea4-819b-4e71-a69f-964b9edc0665 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:23:00.228134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a47ea4-819b-4e71-a69f-964b9edc0665 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:23:00.231641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de989a-22df-4fe7-a371-4761fa07ccc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:23:00.232318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de989a-22df-4fe7-a371-4761fa07ccc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:00.232612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de989a-22df-4fe7-a371-4761fa07ccc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:00.232889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de989a-22df-4fe7-a371-4761fa07ccc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:00.241910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de989a-22df-4fe7-a371-4761fa07ccc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:23:00.243349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de989a-22df-4fe7-a371-4761fa07ccc3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:23:00.253037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 744/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:23:10.163591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=33 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:23:10.248699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=33 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:24:26.358064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.358670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.358670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.379938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:24:26.381186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:26.390600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 750/1494] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:24:26.399941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07863169-4560-4927-9682-395566369635 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.400535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07863169-4560-4927-9682-395566369635 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.400535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07863169-4560-4927-9682-395566369635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.419634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-07863169-4560-4927-9682-395566369635 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:24:26.420884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07863169-4560-4927-9682-395566369635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:26.428883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 745/1495] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:24:26.433515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.443290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9140c0d-e7e1-4ecd-bd21-696ebbf7508a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.448163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.448824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.449638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.450063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.450953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:24:26.451267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:26.452357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:26.452357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:24:26.452357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:24:26.453623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:26.466076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 751/1496] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:24:26.474061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07863169-4560-4927-9682-395566369635 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.483610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07863169-4560-4927-9682-395566369635 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.488450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.489118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.489410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.489683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.490425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:26.492446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:26.492446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:24:26.492446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:24:26.493658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:26.533170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 746/1497] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:24:26 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:24:26.537299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.547960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833a947-f78e-494b-9b5c-bbbda976b556 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.608729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.609312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.609599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.609877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.610585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:26.611597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:26.611597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:24:26.611597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:24:26.612840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:26.634074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 752/1498] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:24:26 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:24:26.639252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.649644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dbbe5-e676-4252-a09a-7bc218bce41b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.654783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.655507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.655757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.656038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.656835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:26.657851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:26.657851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:24:26.657851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:24:26.658738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:26.672306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 747/1499] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:24:26.676574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.687253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276be5-f8dd-4c59-abfc-00049d836834 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.691860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.692561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.692843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.693111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.693834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:26.694829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:26.694829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:24:26.694829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:24:26.695730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17a07f-ffbd-45ff-8f19-fad4db11f83d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:26.710548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 753/1500] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:24:26.715419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.725922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab488c-479f-4713-a730-45fb2b74fb5e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.731162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.732079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.732498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.732781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] PATH_INFO: `/v3/projects/e0270e1a060b4089be7f8752a3350a9a/users/9e0016a8b18b4824bbbd2e2a2d447ad8/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.748329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0270e1a060b4089be7f8752a3350a9a, user_id=9e0016a8b18b4824bbbd2e2a2d447ad8, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:26.749358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:26.749358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:24:26.749358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:24:26.751286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70f21c-81ae-47fb-9cb0-26be83e83d95 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:26.766209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 748/1501] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:24:26 2022] PUT /identity/v3/projects/e0270e1a060b4089be7f8752a3350a9a/users/9e0016a8b18b4824bbbd2e2a2d447ad8/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:24:26.778389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bd72b-00fc-4755-8156-a1c781471458 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.778697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bd72b-00fc-4755-8156-a1c781471458 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.779012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bd72b-00fc-4755-8156-a1c781471458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.806573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-385bd72b-00fc-4755-8156-a1c781471458 None None] MFA Rules not processed for user `9e0016a8b18b4824bbbd2e2a2d447ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:24:26.807799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-385bd72b-00fc-4755-8156-a1c781471458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:26.875735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 754/1502] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:24:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:24:26.889204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6e9d3-81cd-4211-a74d-ca3c39402975 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.889506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6e9d3-81cd-4211-a74d-ca3c39402975 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.889793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6e9d3-81cd-4211-a74d-ca3c39402975 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.891176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 749/1503] 10.222.0.30 () {54 vars in 963 bytes} [Thu Jun 16 18:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:24:26.896279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0413661-81d5-4867-a4c5-d984fd19d2ec None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.896575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0413661-81d5-4867-a4c5-d984fd19d2ec None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.896856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0413661-81d5-4867-a4c5-d984fd19d2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:26.915766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d0413661-81d5-4867-a4c5-d984fd19d2ec None None] MFA Rules not processed for user `49b361e969094db296e3fc02eda18dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:24:26.917017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d0413661-81d5-4867-a4c5-d984fd19d2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:26.963406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:24:26.973079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 755/1504] 10.222.0.30 () {58 vars in 1060 bytes} [Thu Jun 16 18:24:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:24:26.981475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6e9d3-81cd-4211-a74d-ca3c39402975 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:26.994184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6e9d3-81cd-4211-a74d-ca3c39402975 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:26.998616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:26.999279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:26.999561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:26.999836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:27.008863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:27.010347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:27.020601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 750/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:24:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:24:27.062530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e765-e194-44ae-8be8-993031616bce None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:27.062825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e765-e194-44ae-8be8-993031616bce None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:27.063107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e765-e194-44ae-8be8-993031616bce None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:27.064458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 756/1506] 10.222.0.30 () {52 vars in 829 bytes} [Thu Jun 16 18:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:24:27.068208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:27.131232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1ab6b-3366-4936-b26d-32b9f4da7808 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:27.135623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:27.137064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:27.137443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:27.137723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] PATH_INFO: `/v3/users/9e0016a8b18b4824bbbd2e2a2d447ad8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:27.138928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0016a8b18b4824bbbd2e2a2d447ad8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:27.139947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:27.139947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:24:27.139947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:24:27.141336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:27.213789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0016a8b18b4824bbbd2e2a2d447ad8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.227183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0016a8b18b4824bbbd2e2a2d447ad8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.237534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0016a8b18b4824bbbd2e2a2d447ad8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.253908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:24:27.256671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0016a8b18b4824bbbd2e2a2d447ad8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.257527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0016a8b18b4824bbbd2e2a2d447ad8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.266304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 751/1507] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:24:27 2022] DELETE /identity/v3/users/9e0016a8b18b4824bbbd2e2a2d447ad8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:24:27.270714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:27.271004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:27.271278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:27.289515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:24:27.290980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:27.356214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 757/1508] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:24:27.365680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:27.367086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:27.426668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07445504-9154-4f47-8edb-46b3827344cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:27.430337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa570845-1179-4aa1-b975-330fd22036c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:27.431115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570845-1179-4aa1-b975-330fd22036c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:27.431249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570845-1179-4aa1-b975-330fd22036c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:27.431491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570845-1179-4aa1-b975-330fd22036c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:27.441127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa570845-1179-4aa1-b975-330fd22036c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:27.442575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa570845-1179-4aa1-b975-330fd22036c3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:27.452568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 752/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:24:27.795575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:24:27.796721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:24:27.809502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf9473-3715-4a45-b766-aff57a93fca5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:24:27.815133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:24:27.816157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:24:27.816446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:24:27.816744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] PATH_INFO: `/v3/projects/e0270e1a060b4089be7f8752a3350a9a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:24:27.818012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0270e1a060b4089be7f8752a3350a9a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:24:27.819168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:24:27.819168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:24:27.819168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:24:27.820449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:24:27.888098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0270e1a060b4089be7f8752a3350a9a', 'request_id': 'req-66de557f-0cc6-451b-8d3b-208df99c7527'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.901726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0270e1a060b4089be7f8752a3350a9a', 'request_id': 'req-66de557f-0cc6-451b-8d3b-208df99c7527'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.908526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0270e1a060b4089be7f8752a3350a9a', 'request_id': 'req-66de557f-0cc6-451b-8d3b-208df99c7527'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.924502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] The token cache is being invalidate because project e0270e1a060b4089be7f8752a3350a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:24:27.924917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.925551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-66de557f-0cc6-451b-8d3b-208df99c7527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:24:27.927250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 758/1510] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:24:27 2022] DELETE /identity/v3/projects/e0270e1a060b4089be7f8752a3350a9a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:24:37.447934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:24:37.921523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:10.320180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315698-fb9f-46f0-8aae-ba9475166bf4 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:10.320803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315698-fb9f-46f0-8aae-ba9475166bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:10.320803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce315698-fb9f-46f0-8aae-ba9475166bf4 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:10.322133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:10.327148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1c4f5-2f48-4fd7-89e2-db86b70764d6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:10.327617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1c4f5-2f48-4fd7-89e2-db86b70764d6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:10.327722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1c4f5-2f48-4fd7-89e2-db86b70764d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:10.347887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6fb1c4f5-2f48-4fd7-89e2-db86b70764d6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:10.349139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6fb1c4f5-2f48-4fd7-89e2-db86b70764d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:10.414761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:26:10.598084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce315698-fb9f-46f0-8aae-ba9475166bf4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:10.599905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ce315698-fb9f-46f0-8aae-ba9475166bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:10.689040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce315698-fb9f-46f0-8aae-ba9475166bf4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:10.695126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0a214-e088-4b31-b393-ea31880bcce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:10.695838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0a214-e088-4b31-b393-ea31880bcce9 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:10.696163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0a214-e088-4b31-b393-ea31880bcce9 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:10.696452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0a214-e088-4b31-b393-ea31880bcce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:10.707845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0a214-e088-4b31-b393-ea31880bcce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:10.709706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0a214-e088-4b31-b393-ea31880bcce9 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:10.721975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 754/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:11.671908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe81a-06d8-4c49-8789-8a31da62b85d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:11.672782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe81a-06d8-4c49-8789-8a31da62b85d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:11.672782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe81a-06d8-4c49-8789-8a31da62b85d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:11.673942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:26:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:11.679508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca48bb-3e26-4765-9663-8709d281f6f9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:11.680105 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca48bb-3e26-4765-9663-8709d281f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:11.680105 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca48bb-3e26-4765-9663-8709d281f6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:11.698373 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b4ca48bb-3e26-4765-9663-8709d281f6f9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:11.699583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b4ca48bb-3e26-4765-9663-8709d281f6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:11.708180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:26:11.869976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe81a-06d8-4c49-8789-8a31da62b85d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:11.884998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe81a-06d8-4c49-8789-8a31da62b85d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:11.892097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb4a1e-a9c4-4764-a7c5-d1c53da1d64b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:11.892782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb4a1e-a9c4-4764-a7c5-d1c53da1d64b service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:11.893060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb4a1e-a9c4-4764-a7c5-d1c53da1d64b service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:11.893527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb4a1e-a9c4-4764-a7c5-d1c53da1d64b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:11.902812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb4a1e-a9c4-4764-a7c5-d1c53da1d64b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:11.904472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb4a1e-a9c4-4764-a7c5-d1c53da1d64b service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:11.916788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 761/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:26:12.188799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8430e-1892-4cdb-aab3-97d9bf9af78d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:12.189027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8430e-1892-4cdb-aab3-97d9bf9af78d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:12.189264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8430e-1892-4cdb-aab3-97d9bf9af78d None None] PATH_INFO: `/v3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:12.190647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 756/1517] 10.222.0.88 () {54 vars in 925 bytes} [Thu Jun 16 18:26:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:26:12.197079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ecdbd-1779-4b1f-a005-7e1709843c83 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:12.197363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ecdbd-1779-4b1f-a005-7e1709843c83 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:12.197636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ecdbd-1779-4b1f-a005-7e1709843c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:12.216621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c37ecdbd-1779-4b1f-a005-7e1709843c83 None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:12.218473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c37ecdbd-1779-4b1f-a005-7e1709843c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:12.285271 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 762/1518] 10.222.0.88 () {58 vars in 1015 bytes} [Thu Jun 16 18:26:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:26:16.120629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61d7d-aac4-4927-b651-bae6fe0cb579 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:16.121229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61d7d-aac4-4927-b651-bae6fe0cb579 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:16.121229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61d7d-aac4-4927-b651-bae6fe0cb579 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:16.122629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 757/1519] 10.222.0.30 () {54 vars in 960 bytes} [Thu Jun 16 18:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:16.128532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30657a2a-c1a5-45e4-b7ea-978501d0f2af None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:16.129017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30657a2a-c1a5-45e4-b7ea-978501d0f2af None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:16.129116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30657a2a-c1a5-45e4-b7ea-978501d0f2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:16.150319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-30657a2a-c1a5-45e4-b7ea-978501d0f2af None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:16.151698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-30657a2a-c1a5-45e4-b7ea-978501d0f2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:16.161301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 763/1520] 10.222.0.30 () {58 vars in 1057 bytes} [Thu Jun 16 18:26:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:26:16.169842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61d7d-aac4-4927-b651-bae6fe0cb579 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:16.187011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61d7d-aac4-4927-b651-bae6fe0cb579 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:16.192544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186324-7c95-4651-96ac-aa9ebde71974 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:16.193243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186324-7c95-4651-96ac-aa9ebde71974 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:16.193527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186324-7c95-4651-96ac-aa9ebde71974 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:16.193800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186324-7c95-4651-96ac-aa9ebde71974 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:16.205300 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186324-7c95-4651-96ac-aa9ebde71974 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:16.207192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186324-7c95-4651-96ac-aa9ebde71974 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:16.219786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 758/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:23.253806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a5d75-88ef-4984-ad1d-27a12dc2a799 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:23.254425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a5d75-88ef-4984-ad1d-27a12dc2a799 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:23.254425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a5d75-88ef-4984-ad1d-27a12dc2a799 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:23.255661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:23.260827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5392114-7e87-4163-8e5c-32ab683e5fbe None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:23.261291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5392114-7e87-4163-8e5c-32ab683e5fbe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:23.261402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5392114-7e87-4163-8e5c-32ab683e5fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:23.281515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c5392114-7e87-4163-8e5c-32ab683e5fbe None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:23.282762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c5392114-7e87-4163-8e5c-32ab683e5fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:23.291028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:26:23.449302 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a5d75-88ef-4984-ad1d-27a12dc2a799 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:23.465667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a5d75-88ef-4984-ad1d-27a12dc2a799 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:23.472510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62441513-bbb3-4abb-9368-7739cc0ea27c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:23.473162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62441513-bbb3-4abb-9368-7739cc0ea27c service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:23.473447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62441513-bbb3-4abb-9368-7739cc0ea27c service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:23.473724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62441513-bbb3-4abb-9368-7739cc0ea27c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:23.482412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62441513-bbb3-4abb-9368-7739cc0ea27c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:23.484601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62441513-bbb3-4abb-9368-7739cc0ea27c service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:23.494528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 765/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:24.393444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ce36-e9a1-4303-85c2-7850bcef7311 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:24.394053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ce36-e9a1-4303-85c2-7850bcef7311 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:24.394053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ce36-e9a1-4303-85c2-7850bcef7311 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:24.395484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:24.401824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6973a-8d3f-4d47-a7c9-f0df6f5b6796 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:24.402412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6973a-8d3f-4d47-a7c9-f0df6f5b6796 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:24.402563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6973a-8d3f-4d47-a7c9-f0df6f5b6796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:24.433713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bab6973a-8d3f-4d47-a7c9-f0df6f5b6796 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:24.434945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bab6973a-8d3f-4d47-a7c9-f0df6f5b6796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:24.443209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:26:24.598946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0ce36-e9a1-4303-85c2-7850bcef7311 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:24.614048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0ce36-e9a1-4303-85c2-7850bcef7311 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:24.619408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c61bc6-f445-47d2-b00e-55fe9eaae172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:24.620094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c61bc6-f445-47d2-b00e-55fe9eaae172 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:24.621036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c61bc6-f445-47d2-b00e-55fe9eaae172 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:24.621313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c61bc6-f445-47d2-b00e-55fe9eaae172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:24.631396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c61bc6-f445-47d2-b00e-55fe9eaae172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:24.633037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c61bc6-f445-47d2-b00e-55fe9eaae172 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:24.642601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 761/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:28.839144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfcf0e-e8e8-450c-837c-1b5ec822fec5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:28.839724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfcf0e-e8e8-450c-837c-1b5ec822fec5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:28.839724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfcf0e-e8e8-450c-837c-1b5ec822fec5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:28.841160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:28.846704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba278b4-c35d-4445-a7c9-4d3ef314fd49 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:28.847196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba278b4-c35d-4445-a7c9-4d3ef314fd49 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:28.847305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba278b4-c35d-4445-a7c9-4d3ef314fd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:28.865551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6ba278b4-c35d-4445-a7c9-4d3ef314fd49 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:28.866763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6ba278b4-c35d-4445-a7c9-4d3ef314fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:28.875310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:26:28.912140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfcf0e-e8e8-450c-837c-1b5ec822fec5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:28.913852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b4cfcf0e-e8e8-450c-837c-1b5ec822fec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:28.975928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfcf0e-e8e8-450c-837c-1b5ec822fec5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:28.979575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c19a6-07cb-4d38-9ace-81bf4637b383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:28.980265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c19a6-07cb-4d38-9ace-81bf4637b383 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:28.980553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c19a6-07cb-4d38-9ace-81bf4637b383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:28.980832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c19a6-07cb-4d38-9ace-81bf4637b383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:28.991130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c19a6-07cb-4d38-9ace-81bf4637b383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:28.992697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c19a6-07cb-4d38-9ace-81bf4637b383 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:28.997907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:29.004298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 768/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:31.175892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42605-7419-48cd-bd2e-1c8e37c0fd91 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:31.176599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42605-7419-48cd-bd2e-1c8e37c0fd91 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:31.176599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42605-7419-48cd-bd2e-1c8e37c0fd91 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:31.177884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:31.183578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b407-caaa-4250-b4af-a7abe3026395 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:31.184031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b407-caaa-4250-b4af-a7abe3026395 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:31.184168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703b407-caaa-4250-b4af-a7abe3026395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:31.202810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b703b407-caaa-4250-b4af-a7abe3026395 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:31.204029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b703b407-caaa-4250-b4af-a7abe3026395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:31.212291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:26:31.249418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42605-7419-48cd-bd2e-1c8e37c0fd91 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:31.260324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42605-7419-48cd-bd2e-1c8e37c0fd91 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:31.264560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7b9d2-74d8-4699-9f4d-769e217685ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:31.265290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7b9d2-74d8-4699-9f4d-769e217685ab service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:31.265568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7b9d2-74d8-4699-9f4d-769e217685ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:31.265837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7b9d2-74d8-4699-9f4d-769e217685ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:31.275226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7b9d2-74d8-4699-9f4d-769e217685ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:31.276698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7b9d2-74d8-4699-9f4d-769e217685ab service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:31.286426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 764/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:35.052666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a3e40-57f4-4026-bc07-8d0f0c44f23c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:35.053297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a3e40-57f4-4026-bc07-8d0f0c44f23c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:35.053404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a3e40-57f4-4026-bc07-8d0f0c44f23c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:35.054874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:35.060883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d154-2411-4b1d-a546-9c7586055870 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:35.061359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d154-2411-4b1d-a546-9c7586055870 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:35.061468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d154-2411-4b1d-a546-9c7586055870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:35.081504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7a09d154-2411-4b1d-a546-9c7586055870 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:35.082720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7a09d154-2411-4b1d-a546-9c7586055870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:35.090751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:26:35.358501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a3e40-57f4-4026-bc07-8d0f0c44f23c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:35.360236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-da7a3e40-57f4-4026-bc07-8d0f0c44f23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:35.423602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a3e40-57f4-4026-bc07-8d0f0c44f23c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:35.427175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548eb5f-0449-45f6-b70a-8dc96bce4b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:35.427818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548eb5f-0449-45f6-b70a-8dc96bce4b12 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:35.428120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548eb5f-0449-45f6-b70a-8dc96bce4b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:35.428464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548eb5f-0449-45f6-b70a-8dc96bce4b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:35.439822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548eb5f-0449-45f6-b70a-8dc96bce4b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:35.441337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548eb5f-0449-45f6-b70a-8dc96bce4b12 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:35.450947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 771/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:37.648853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45cb79-5f30-42a5-8eb3-c4c49349cf72 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:37.649454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45cb79-5f30-42a5-8eb3-c4c49349cf72 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:37.649454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45cb79-5f30-42a5-8eb3-c4c49349cf72 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:37.650830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:26:37.656632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e51f10-2def-4dc2-9290-19147ae4279a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:37.657098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e51f10-2def-4dc2-9290-19147ae4279a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:37.657210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e51f10-2def-4dc2-9290-19147ae4279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:37.676217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-77e51f10-2def-4dc2-9290-19147ae4279a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:26:37.677434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-77e51f10-2def-4dc2-9290-19147ae4279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:37.685744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:26:37.971589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45cb79-5f30-42a5-8eb3-c4c49349cf72 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:37.973356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5f45cb79-5f30-42a5-8eb3-c4c49349cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:37.984825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=65 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:37.991307 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45cb79-5f30-42a5-8eb3-c4c49349cf72 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:37.995224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eaf90-35f3-458d-9d46-ca54dd296955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:37.995918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eaf90-35f3-458d-9d46-ca54dd296955 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:37.996252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eaf90-35f3-458d-9d46-ca54dd296955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:37.996574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eaf90-35f3-458d-9d46-ca54dd296955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:38.007627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59eaf90-35f3-458d-9d46-ca54dd296955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:38.009110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59eaf90-35f3-458d-9d46-ca54dd296955 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:38.020181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 767/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:26:47.670710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=32 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:48.015778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:16.642365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:16.643026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:16.643026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:16.671827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:16.673514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:16.686497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 773/1540] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:16.697273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:16.698435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:16.698932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:16.729696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:16.731916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:16.745488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 768/1541] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:28:16.751898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:16.767291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c1342-9f2a-493b-bf23-c7dedb706fff None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:16.776578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:16.778038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:16.778758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:16.779406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:16.780541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:16.781097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:16.783212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:16.803673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 774/1542] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:16.813672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:16.827333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077bed7-9c95-4153-8870-3d0a03ad26db None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:16.834957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:16.836076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:16.836750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:16.837249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:16.838524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:16.841760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:16.900919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 769/1543] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:16 2022] POST /identity/v3/projects => generated 355 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:16.906700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:16.923700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c34a-ef43-4bf4-b4fb-08365fba7ffc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.018048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296d9c9-f151-4dd0-9c5d-d9d8f819679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.019286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296d9c9-f151-4dd0-9c5d-d9d8f819679a admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.020346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296d9c9-f151-4dd0-9c5d-d9d8f819679a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.020902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296d9c9-f151-4dd0-9c5d-d9d8f819679a admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.022442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296d9c9-f151-4dd0-9c5d-d9d8f819679a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.029393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296d9c9-f151-4dd0-9c5d-d9d8f819679a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.065966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 775/1544] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:16 2022] POST /identity/v3/users => generated 278 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:17.071405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.088515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652d1b2-9cf3-456c-9655-afb4c9096e1b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.088848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652d1b2-9cf3-456c-9655-afb4c9096e1b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.089352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652d1b2-9cf3-456c-9655-afb4c9096e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.109193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d470d3-ea6d-432d-9618-b27253f30f84 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.123060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7652d1b2-9cf3-456c-9655-afb4c9096e1b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.124540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7652d1b2-9cf3-456c-9655-afb4c9096e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.129134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad34e-0248-4401-9723-3ce88c5826a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.130676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad34e-0248-4401-9723-3ce88c5826a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.131855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad34e-0248-4401-9723-3ce88c5826a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.132722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad34e-0248-4401-9723-3ce88c5826a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.134342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ad34e-0248-4401-9723-3ce88c5826a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.136990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ad34e-0248-4401-9723-3ce88c5826a8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.137889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 776/1545] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.139640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.140217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.140540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.173516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.175089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.179415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 770/1546] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:28:17.182811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54d6ad-3b9f-4f0b-bbfc-b5fe26c480c9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.188584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54d6ad-3b9f-4f0b-bbfc-b5fe26c480c9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.189703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54d6ad-3b9f-4f0b-bbfc-b5fe26c480c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.190891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 777/1547] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.191433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.216927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139818-3da1-4d17-9fe4-719dcb3136db None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.229515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7277bcb-8b9d-438b-8e8d-6d25c7ee40e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.230997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277bcb-8b9d-438b-8e8d-6d25c7ee40e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.231288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277bcb-8b9d-438b-8e8d-6d25c7ee40e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.231622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277bcb-8b9d-438b-8e8d-6d25c7ee40e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.240087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7277bcb-8b9d-438b-8e8d-6d25c7ee40e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.240087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7277bcb-8b9d-438b-8e8d-6d25c7ee40e7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.259909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 778/1548] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:17.261359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.262120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.262304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.263958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-af54d6ad-3b9f-4f0b-bbfc-b5fe26c480c9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.266501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-af54d6ad-3b9f-4f0b-bbfc-b5fe26c480c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.297749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 771/1549] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.300112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.304879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.306685 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1183fd8-d6a8-46a6-9abd-f0a5cbb2f1c5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.307567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1183fd8-d6a8-46a6-9abd-f0a5cbb2f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.308381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1183fd8-d6a8-46a6-9abd-f0a5cbb2f1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.344435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 779/1550] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.349909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.371032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f1183fd8-d6a8-46a6-9abd-f0a5cbb2f1c5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.375011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f1183fd8-d6a8-46a6-9abd-f0a5cbb2f1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.391946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05307f49-5ec0-4d0d-9bc7-789b107d25fc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.413553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 772/1551] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.414713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.416525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.417284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.417787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] PATH_INFO: `/v3/projects/a9353f1b4e36450eb953f2171c4dab8a/users/1a2db1a1cdd34ba09b1cf42c9275fa3c/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.423552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.424085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.424610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.461675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.463059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.478965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9353f1b4e36450eb953f2171c4dab8a, user_id=1a2db1a1cdd34ba09b1cf42c9275fa3c, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.488625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:28:17.488625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:28:17.488625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:28:17.512095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.517369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 773/1552] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.518959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.555594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020db1f-3073-427a-88c3-1cf67b431281 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.577727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 780/1553] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:17 2022] PUT /identity/v3/projects/a9353f1b4e36450eb953f2171c4dab8a/users/1a2db1a1cdd34ba09b1cf42c9275fa3c/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:17.579480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.681281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.682230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b75fc-3cd4-4037-8963-f8d73b1247fd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.682729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.682825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.683650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.686258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:17.686258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.687123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:28:17.687123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:28:17.687123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:28:17.688999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72752359-999c-4427-9d8c-64b0fa8c4248 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.693039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.694018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.694497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.695307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.696675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:17.697194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.700279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4e70-7d51-4b7b-8f55-0e7f41a19e1d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.709067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 774/1554] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:17.711043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c20375-586d-415c-865e-35c0b8fb5cb3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.711684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c20375-586d-415c-865e-35c0b8fb5cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.712216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c20375-586d-415c-865e-35c0b8fb5cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.723578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 781/1555] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:17.727405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006f7ab-f515-4f5b-973a-7f2b59af9faa None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.727903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006f7ab-f515-4f5b-973a-7f2b59af9faa None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.728442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006f7ab-f515-4f5b-973a-7f2b59af9faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.739544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-53c20375-586d-415c-865e-35c0b8fb5cb3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.741235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-53c20375-586d-415c-865e-35c0b8fb5cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.751176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 775/1556] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.753297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df16a-a1d2-4e0a-bea9-e4d351c412dc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.753753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df16a-a1d2-4e0a-bea9-e4d351c412dc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.754203 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df16a-a1d2-4e0a-bea9-e4d351c412dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.755825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6006f7ab-f515-4f5b-973a-7f2b59af9faa None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.758117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6006f7ab-f515-4f5b-973a-7f2b59af9faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.771895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 782/1557] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.773685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c142c0-2f73-4345-ab70-560f53de7320 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.774319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c142c0-2f73-4345-ab70-560f53de7320 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.774603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c142c0-2f73-4345-ab70-560f53de7320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.782653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:17.794376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5a8df16a-a1d2-4e0a-bea9-e4d351c412dc None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.796194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5a8df16a-a1d2-4e0a-bea9-e4d351c412dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.801431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-71c142c0-2f73-4345-ab70-560f53de7320 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.803661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-71c142c0-2f73-4345-ab70-560f53de7320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.809762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 776/1558] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.811996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.812436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.812901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.830615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 783/1559] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.830615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.830615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.830615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.858127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] MFA Rules not processed for user `1a2db1a1cdd34ba09b1cf42c9275fa3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.859598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.860118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:17.861280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:17.884524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 784/1560] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.885959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.904579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf35d70-da38-4740-af54-9a6fdb64357c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:17.911543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:17.912603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:17.913039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:17.913451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:17.914581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:17.916379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:28:17.916379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:28:17.916379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:28:17.919214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:17.964476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 785/1561] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:17.964476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.973034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 777/1562] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:17.974305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:17.987457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490c6d8-c152-43d7-a5fc-b111d0a4c7ca None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.002866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:18.072654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6446503-ef37-4755-a8cb-e4e2ce88a851 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.075820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.076639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.077022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.077429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.078468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:18.078877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.079329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.080209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.080674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.081046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.081647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801da404-0d1f-4950-a764-1c5666294500 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.081891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.084168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.106790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 778/1563] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:18.106790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fad83-1017-4016-9121-5ecb937adb89 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.106790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fad83-1017-4016-9121-5ecb937adb89 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.106790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fad83-1017-4016-9121-5ecb937adb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.130303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-654fad83-1017-4016-9121-5ecb937adb89 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:18.130856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 786/1564] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:17 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.132076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-654fad83-1017-4016-9121-5ecb937adb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:18.132527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.229929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 779/1565] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:18.232930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.233424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.233868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.243254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74c1b-548b-4c15-807c-f013b199b006 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.251873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.253000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.253452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.253895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.255154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:18.255663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.258622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8d725-cca1-48e2-8fe2-803a88be3db1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.268185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:18.270238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:18.288594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 787/1566] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:18.288818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 780/1567] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:18.288932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.288932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.288932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.290014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.302407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3959f-0a6a-4588-be29-bb23008d82cd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.307903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.309870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.310326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.310615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.311342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.312452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:28:18.312452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:28:18.312452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:28:18.313722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.314640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:18.315978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:18.327285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 788/1568] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:18.328766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.342177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d418930-c0a6-48ae-af9d-1db261c3ac6a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.343730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 781/1569] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.344411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.350966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.351915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.352358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.352793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.353742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.355858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.357867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53a2c-1057-422a-b4f4-9edb1c39e70d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.366861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.368075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.368628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.369127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.370515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.375044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.389162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 789/1570] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/projects => generated 355 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.390245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.420699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 782/1571] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.421809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.428672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:18.508190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fc1f7-c723-4c09-9a39-f4ac9891993f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.512214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e123-d787-4ef2-94b0-f795ce4c7a2f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.517101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.518307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.518815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.519321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.519460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.520299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.520576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.520781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.521157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.522035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:18.522455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.524273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.524443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.526451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:18.544029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 783/1572] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:18.545119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.557080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c518-3fa2-4937-8bc1-4489ff0fc29d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.564097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.565452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.565966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.566234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.567072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:18.567386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.569079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.574235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 790/1573] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.575553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.584402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 784/1574] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:18.585299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.592441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6a8e-747c-4d4b-8f4c-663be55a340f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.598230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35cfca-b453-464e-9c50-802857480e64 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.600446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.601683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.602055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.602514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.603739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.606822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.607566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.607847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.608097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.608476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.609708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:18.610228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.613228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.637178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 785/1575] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:18.639290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.650208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 791/1576] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/projects => generated 393 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.651881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.659361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ddea2-0c72-45d3-bbe3-4e33e13cea5f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.735558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b0d64-ceef-4fbf-98e1-dd666cae2d2d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.741096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.741948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.742232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.742504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.743393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.745411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.762076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.763463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.763975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.764687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.766061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.769475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.770487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 792/1577] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.771489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.786525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 786/1578] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:18.787224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.787406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136826d-d35c-4e4f-aa8f-3e980cfdd61b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.797403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.798507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.798961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.799453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.800801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.804240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.807171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf936-86d4-47ac-beba-580863bffe0b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.815360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.816720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.817195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.817637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.818963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.821950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.839448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 793/1579] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/users => generated 284 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.841664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.858749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3d588-3b57-4830-b0ea-fb9e21a0c989 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.864672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.865729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.866247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.866673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.867721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.869249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 787/1580] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:18.874084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.874240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.885939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 794/1581] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:18.887364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:18.899290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042963ff-ac01-422c-8cff-897d4d0573a6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.959465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:18.972146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413574de-72f0-4da0-bff5-808311e3ba5f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:18.978364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.979078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.979361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.979635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.981080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.981342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:18.982181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:18.982645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:18.983194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:18.984201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:18.984538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:18.988400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.017597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 795/1582] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:19.018586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.022616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 788/1583] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:18 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:19.023528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.060754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:19.100686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8bd43-2ac5-451c-a9cd-2c82348c34ca admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.119474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4cbf-9539-4221-b145-480b5079b5f2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.127747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.128968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.129464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.129958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.131370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.134624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.156129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 789/1584] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.157488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.172931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.173814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.174203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.174503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.175398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.176532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f66adf-b376-4471-af5b-52217e8ee592 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.178220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.183404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.184305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.184710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.185021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.185939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.187776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.204463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 790/1585] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.205736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.208940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 796/1586] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:19 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:19.210564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.223462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e9aa7-d22b-460f-8930-821d1248cd8c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.223768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914de9f-d5fb-4ee0-b39e-0818401163dd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.229245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.229590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.230191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.230438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.230646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.231009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.231333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.231483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.232222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.232443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.234272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.234535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.249242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 791/1587] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.250762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.265001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 797/1588] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:19 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:19.265348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2174-54b1-40d4-b25a-666ea7860ca7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.270131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.272003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.273023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.273497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.273933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.274828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.276931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.285978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd1d4c-01f8-45ea-99bb-2e715c0c2140 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.293746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.294472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.294799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.295051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.295596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 792/1589] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.295778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.296801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.298585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.311716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77627c7-0762-4ff8-99a3-8e4577681469 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.319970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.320870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.321237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.321687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.322521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.325498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.330188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 798/1590] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:19 2022] POST /identity/v3/users => generated 295 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:19.330188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.347072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1ce85-d102-44d2-8f8d-16f95df5bfd7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.355032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.357050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.357498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.357944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] PATH_INFO: `/v3/projects/133bf8c4b0684602b2b7b1f9549f17bb/users/af98332fe9fa4c1d9055709561939919/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.364032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 793/1591] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:19 2022] POST /identity/v3/users => generated 289 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:19.364032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.375855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f2e9c-e0a7-4ffa-9849-cbceed01c6e7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.383030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.384720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.385180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.385627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] PATH_INFO: `/v3/projects/1f0473c6214a4428be438fabc530012b/users/89f6c755dbf74640a372802f02733eb3/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.389228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133bf8c4b0684602b2b7b1f9549f17bb, user_id=af98332fe9fa4c1d9055709561939919, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.393778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.411826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0473c6214a4428be438fabc530012b, user_id=89f6c755dbf74640a372802f02733eb3, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.415460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.421268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 799/1592] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:19 2022] PUT /identity/v3/projects/133bf8c4b0684602b2b7b1f9549f17bb/users/af98332fe9fa4c1d9055709561939919/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:19.422897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.440538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 794/1593] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:19 2022] PUT /identity/v3/projects/1f0473c6214a4428be438fabc530012b/users/89f6c755dbf74640a372802f02733eb3/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:19.442223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:19.447180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.531724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6b84-480b-4296-bd03-9c245850ed88 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.559429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a256b9-ebda-4501-be3e-25e700cfbe43 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.568023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.568948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.569346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.569797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.570866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.573035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.585663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 795/1594] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.588983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.601625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e436d3-5f6a-46c3-9323-aff79f7980d8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.608897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.609432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.609715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.609984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.610763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.612562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.625215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 796/1595] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.626081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.634792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.635828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.636341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.637288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.638592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.638815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886301-f63b-409e-bc92-0d56ab6796ae admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.640841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.643878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.644832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.645111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.645385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] PATH_INFO: `/v3/projects/51d24550b26342efa6a0a51f02b42627/users/be1a243261e94da2af29163669af57e7/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.655863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 800/1596] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.657473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.669840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d24550b26342efa6a0a51f02b42627, user_id=be1a243261e94da2af29163669af57e7, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.672959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.673443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ed223-5fbf-4a2a-8ba1-ef59b5a4c0d1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.679561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.680679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.681043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.681540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.682835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.685492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.695349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 797/1597] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:19 2022] PUT /identity/v3/projects/51d24550b26342efa6a0a51f02b42627/users/be1a243261e94da2af29163669af57e7/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:19.696994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.705418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 801/1598] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.706335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.717692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:19.722512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf32c09-7ea9-4b0a-ba39-7cd94cf11653 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.805682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c6485-4d2e-4434-81e9-37cd1bc425ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.807375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bde2a-3450-476b-8d96-fe5dd5bd9ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.809172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bde2a-3450-476b-8d96-fe5dd5bd9ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.809399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bde2a-3450-476b-8d96-fe5dd5bd9ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.809856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bde2a-3450-476b-8d96-fe5dd5bd9ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.811319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bde2a-3450-476b-8d96-fe5dd5bd9ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.814428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bde2a-3450-476b-8d96-fe5dd5bd9ba7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.821582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.822650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.823110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.823562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.825179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.828103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.834320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 802/1599] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.836342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.836801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.837249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.848351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 798/1600] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.849410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.865835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511aa16-7bd9-4aa1-855a-61c901588e38 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.866660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:19.867973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:19.873191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.874370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.874816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.875255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] PATH_INFO: `/v3/projects/7bca05f0fa1b4e55b61cfa6991aceea1/users/7a4d1e0b870f472fbd2648f0d5f45e03/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.878181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 803/1601] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:19.879059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.895333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9e33-b558-4954-aa5f-4d4a4349025d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.900916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.901884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.903062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.903062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.903801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.905541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.915586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bca05f0fa1b4e55b61cfa6991aceea1, user_id=7a4d1e0b870f472fbd2648f0d5f45e03, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.918166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 804/1602] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.919235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.921599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.932249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c67ff-a079-42a1-beda-c45ec286b548 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:19.938902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5812b-8172-4172-956e-1de02466bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:19.939653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5812b-8172-4172-956e-1de02466bfad admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.939940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5812b-8172-4172-956e-1de02466bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.940309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5812b-8172-4172-956e-1de02466bfad admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:19.941126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5812b-8172-4172-956e-1de02466bfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:19.942795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5812b-8172-4172-956e-1de02466bfad admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:19.944736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 799/1603] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:19 2022] PUT /identity/v3/projects/7bca05f0fa1b4e55b61cfa6991aceea1/users/7a4d1e0b870f472fbd2648f0d5f45e03/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:19.946050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:19.970404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 805/1604] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:19.973328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:19.973691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:19.974322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.006293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:20.018008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] MFA Rules not processed for user `be1a243261e94da2af29163669af57e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:20.019241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:20.068979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57884360-4dce-43c2-b936-243e350b3367 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.078222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.079393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.079889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.080434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.081852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.085149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.102856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 806/1605] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:20.103741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.110950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 800/1606] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:20.110950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.118624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd504d1-d50f-468d-a59b-d6747d742459 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.123625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.124594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.124872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.125143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] PATH_INFO: `/v3/projects/133bf8c4b0684602b2b7b1f9549f17bb/users/af98332fe9fa4c1d9055709561939919/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.127288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=133bf8c4b0684602b2b7b1f9549f17bb, user_id=af98332fe9fa4c1d9055709561939919, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.128239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29f4b-02b0-40ae-a304-c1e68fa0d4fb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.130552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.135616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.137656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.137656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.138063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.139130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.141276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.146352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 807/1607] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:20 2022] PUT /identity/v3/projects/133bf8c4b0684602b2b7b1f9549f17bb/users/af98332fe9fa4c1d9055709561939919/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:20.147239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.169422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 801/1608] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:20.169422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.179763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6299be-bb42-41ac-a267-e0598154fb35 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.228303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b33b9-260c-4e9a-8975-858816da1ffa admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.234560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.235297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.235637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.236038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.268344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e51a1-c9ce-46bc-ab21-6e08cf74cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.270325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e51a1-c9ce-46bc-ab21-6e08cf74cac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.270614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e51a1-c9ce-46bc-ab21-6e08cf74cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.271076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e51a1-c9ce-46bc-ab21-6e08cf74cac2 admin admin] PATH_INFO: `/v3/projects/a367968c80bb4ce9973d18c1674c01b1/users/bc80a29a78af4201becf229b0a964fa3/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.287471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:20.296906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e51a1-c9ce-46bc-ab21-6e08cf74cac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a367968c80bb4ce9973d18c1674c01b1, user_id=bc80a29a78af4201becf229b0a964fa3, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.301365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e51a1-c9ce-46bc-ab21-6e08cf74cac2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.321030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 802/1609] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:20 2022] PUT /identity/v3/projects/a367968c80bb4ce9973d18c1674c01b1/users/bc80a29a78af4201becf229b0a964fa3/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:20.322659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.323100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.323552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.345499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.348753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.363504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 808/1610] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:20.364947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.367692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] MFA Rules not processed for user `7a4d1e0b870f472fbd2648f0d5f45e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:20.369945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:20.385232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880f3bb-5f2f-4d4b-8d80-468b80881c84 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.451241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:20.512792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.514895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.516263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.517190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] PATH_INFO: `/v3/projects/b70bbccf07e3404cb94ba59201061a6a/users/512f01cca14b447c8f6c9eebbd828eb2/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.527478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 803/1611] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:20.528469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.550543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4e9fc-7bf7-4c93-92d5-4274018f1dc5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.553724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70bbccf07e3404cb94ba59201061a6a, user_id=512f01cca14b447c8f6c9eebbd828eb2, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.557108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709aaa4a-0ef6-4418-887e-82b79bdabcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.557884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aaa4a-0ef6-4418-887e-82b79bdabcbc admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.559042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.559240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aaa4a-0ef6-4418-887e-82b79bdabcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.559744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aaa4a-0ef6-4418-887e-82b79bdabcbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.561247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709aaa4a-0ef6-4418-887e-82b79bdabcbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.563662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709aaa4a-0ef6-4418-887e-82b79bdabcbc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.584102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 809/1612] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:20 2022] PUT /identity/v3/projects/b70bbccf07e3404cb94ba59201061a6a/users/512f01cca14b447c8f6c9eebbd828eb2/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:20.585034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.615399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 804/1613] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:20 2022] POST /identity/v3/projects => generated 363 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:20.618057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.618807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.619537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.681677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] MFA Rules not processed for user `af98332fe9fa4c1d9055709561939919`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:20.684487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:20.727478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efd372-4765-4d1b-b22b-267a989ea070 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.808127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 805/1614] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:20.808127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:20.848401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3649e-7726-4970-894c-c0c42eff55eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:20.850619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3649e-7726-4970-894c-c0c42eff55eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.850897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3649e-7726-4970-894c-c0c42eff55eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.851175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3649e-7726-4970-894c-c0c42eff55eb admin admin] PATH_INFO: `/v3/projects/7904e707e39846fdb8b7d019719dc163/users/73dc3ab6bc6448a89b26e3e1a1b5c6e8/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.891450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3649e-7726-4970-894c-c0c42eff55eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7904e707e39846fdb8b7d019719dc163, user_id=73dc3ab6bc6448a89b26e3e1a1b5c6e8, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:20.897325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3649e-7726-4970-894c-c0c42eff55eb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:20.901374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:20.929607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 810/1615] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:20 2022] PUT /identity/v3/projects/7904e707e39846fdb8b7d019719dc163/users/73dc3ab6bc6448a89b26e3e1a1b5c6e8/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:20.932295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25cb67-18ca-4919-aada-344f148049c7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:20.933054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25cb67-18ca-4919-aada-344f148049c7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:20.933054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25cb67-18ca-4919-aada-344f148049c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:20.938451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c842e5-cd92-4c6e-8b51-557eaa6ebcdf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:20.992034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fb25cb67-18ca-4919-aada-344f148049c7 None None] MFA Rules not processed for user `bc80a29a78af4201becf229b0a964fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:20.994194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fb25cb67-18ca-4919-aada-344f148049c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:21.040411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.041493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.042133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.042516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] PATH_INFO: `/v3/projects/aa9111175cad44b2bbc611b7ae36a991/users/7ee9c06c774f478db33c16b784d6b914/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.061359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:21.073518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9111175cad44b2bbc611b7ae36a991, user_id=7ee9c06c774f478db33c16b784d6b914, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.077304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.095375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 806/1616] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:20 2022] PUT /identity/v3/projects/aa9111175cad44b2bbc611b7ae36a991/users/7ee9c06c774f478db33c16b784d6b914/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:21.096590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.117642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 811/1617] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:21.119813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.119813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.120393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.176864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:21.179314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:21.211945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049508-6ab4-4085-933d-c2079c68e214 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.218012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.218713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.219073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.219354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.318336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 812/1618] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:21.319954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.336998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8d6ca-4c8a-490d-8035-2f2efb8e849a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.338868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.342663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.345686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ec132-2224-4849-9c5b-99f32ed7f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.347210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ec132-2224-4849-9c5b-99f32ed7f820 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.347993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ec132-2224-4849-9c5b-99f32ed7f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.348599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ec132-2224-4849-9c5b-99f32ed7f820 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.350280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ec132-2224-4849-9c5b-99f32ed7f820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.357016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ec132-2224-4849-9c5b-99f32ed7f820 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.362756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 807/1619] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:21.366271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.384908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c9462-50e6-47a8-8a80-25410eac0108 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.391890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.392904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.393495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.393891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.412801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 813/1620] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:21 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:21.415077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c8ce-358b-49d5-bf97-e0a45ed9f293 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.415495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c8ce-358b-49d5-bf97-e0a45ed9f293 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.415948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c8ce-358b-49d5-bf97-e0a45ed9f293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.443451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c823c8ce-358b-49d5-bf97-e0a45ed9f293 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:21.445678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c823c8ce-358b-49d5-bf97-e0a45ed9f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:21.457146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:21.466527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 814/1621] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:21.466527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0d75f-ed28-4bc1-820b-2495c190874e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.466527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0d75f-ed28-4bc1-820b-2495c190874e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.466527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0d75f-ed28-4bc1-820b-2495c190874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.532805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1ea0d75f-ed28-4bc1-820b-2495c190874e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:21.537011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ea0d75f-ed28-4bc1-820b-2495c190874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:21.553934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 815/1622] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:21.555441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f256f4-2550-484e-9fa4-76c3f3b4fb57 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.555737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f256f4-2550-484e-9fa4-76c3f3b4fb57 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.556014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f256f4-2550-484e-9fa4-76c3f3b4fb57 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.558032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 816/1623] 10.222.0.30 () {54 vars in 961 bytes} [Thu Jun 16 18:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:28:21.567337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.569352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.575312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f256f4-2550-484e-9fa4-76c3f3b4fb57 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.591252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 808/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:21.592702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.613056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8904771-d0fa-458d-a31a-ef1fbefc4717 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.619319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.620184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.620918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.622916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.623836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.626094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.642436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 809/1625] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:21.643463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.737527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f256f4-2550-484e-9fa4-76c3f3b4fb57 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.740922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b0de6-f29c-44e7-abf2-a6c1cfe2cce0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.741918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.742621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.742889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.743157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.746301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53905cf8-ca01-4944-8599-18f290962361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.747800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53905cf8-ca01-4944-8599-18f290962361 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.748116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53905cf8-ca01-4944-8599-18f290962361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.748403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53905cf8-ca01-4944-8599-18f290962361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.759831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.760343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53905cf8-ca01-4944-8599-18f290962361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.763479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53905cf8-ca01-4944-8599-18f290962361 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.770160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.781546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 810/1626] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:21.783027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.783320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.783635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.791610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 817/1627] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:21.794163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.801392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:21.828796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:21.830077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:21.830478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7611d9-3dca-4e59-a57d-62fba502c23b service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.841465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.842147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.842428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.842726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.875560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:21.877110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:21.914022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 818/1628] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:21.915077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.937275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 811/1629] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Jun 16 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:21.953689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96314-68d2-4f55-a2e5-0a761ec4de08 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.958177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:21.967219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.968026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.968334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.968607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] PATH_INFO: `/v3/projects/7e6ebb052f3b4a22b1ef9cf83d842c0b/users/f33ca4bf35634adfa5dad8a3627dedf4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:21.972733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fa8bf-9d5f-446c-aff8-e9412e22c70e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:21.987773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:21.989294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:21.989591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:21.989920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:22.048831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ebb052f3b4a22b1ef9cf83d842c0b, user_id=f33ca4bf35634adfa5dad8a3627dedf4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:22.051621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:22.085463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 819/1630] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:21 2022] PUT /identity/v3/projects/7e6ebb052f3b4a22b1ef9cf83d842c0b/users/f33ca4bf35634adfa5dad8a3627dedf4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:22.086586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:22.092964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:22.123775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:22.126687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:22.129982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:22.155684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 812/1631] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:28:22.159457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:22.178729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af534-802b-4fcd-b302-66c401b1ac87 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:22.234579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395bb5-301b-4d06-aae6-b29bcfb666eb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:22.239792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:22.240662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:22.240948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:22.241424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:22.279017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:22.280603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:22.281154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:22.282360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:22.283431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:22.287028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:22.303731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 813/1632] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:22.308088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:22.329981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75638424-11df-4bb3-9350-85fcbc79219e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:22.340086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36028c-0a53-4965-bcb8-eeace97024af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:22.343054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36028c-0a53-4965-bcb8-eeace97024af admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:22.343504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36028c-0a53-4965-bcb8-eeace97024af admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:22.343934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36028c-0a53-4965-bcb8-eeace97024af admin admin] PATH_INFO: `/v3/projects/7e6ebb052f3b4a22b1ef9cf83d842c0b/users/f33ca4bf35634adfa5dad8a3627dedf4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:22.347406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36028c-0a53-4965-bcb8-eeace97024af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ebb052f3b4a22b1ef9cf83d842c0b, user_id=f33ca4bf35634adfa5dad8a3627dedf4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:22.351718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36028c-0a53-4965-bcb8-eeace97024af admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:22.385719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 814/1633] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:22 2022] PUT /identity/v3/projects/7e6ebb052f3b4a22b1ef9cf83d842c0b/users/f33ca4bf35634adfa5dad8a3627dedf4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:28:22.389398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:22.390064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:22.391620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:22.446583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:22.448596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:22.462941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 820/1634] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:22 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:22.539321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] MFA Rules not processed for user `f33ca4bf35634adfa5dad8a3627dedf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:22.541617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:22.834380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 815/1635] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:22.864115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:22.889185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8330da-abea-4a47-8f73-34f7fdb02501 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:23.104425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b95444-036d-4363-8c1a-a25b47e23953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:23.106155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b95444-036d-4363-8c1a-a25b47e23953 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:23.106638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b95444-036d-4363-8c1a-a25b47e23953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:23.107105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b95444-036d-4363-8c1a-a25b47e23953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:23.151758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b95444-036d-4363-8c1a-a25b47e23953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:23.159387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b95444-036d-4363-8c1a-a25b47e23953 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:23.208346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 821/1636] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:23.579811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:23.598934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb6888-6341-4cd9-9c5c-e0c172c5cf42 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:23.610447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6559fba-7165-4252-88c4-50cb8405b9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:23.611490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6559fba-7165-4252-88c4-50cb8405b9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:23.612110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6559fba-7165-4252-88c4-50cb8405b9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:23.613679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6559fba-7165-4252-88c4-50cb8405b9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:23.781161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:23.800209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6559fba-7165-4252-88c4-50cb8405b9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:23.802139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6559fba-7165-4252-88c4-50cb8405b9b9 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:23.816571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 816/1637] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:23.906886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d972c-4bbc-443b-bb34-a1f9fc18b025 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:23.907512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d972c-4bbc-443b-bb34-a1f9fc18b025 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:23.908023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d972c-4bbc-443b-bb34-a1f9fc18b025 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:23.911385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 822/1638] 10.222.0.30 () {54 vars in 961 bytes} [Thu Jun 16 18:28:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:28:23.923240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c3c87-d820-478b-a085-d7593b89aa0e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:23.924250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c3c87-d820-478b-a085-d7593b89aa0e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:23.924250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c3c87-d820-478b-a085-d7593b89aa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:23.954305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bb4c3c87-d820-478b-a085-d7593b89aa0e None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:23.956239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bb4c3c87-d820-478b-a085-d7593b89aa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:24.152716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 817/1639] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Jun 16 18:28:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:28:24.164168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce824-2bdf-4434-87da-574a206a3994 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:24.164546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce824-2bdf-4434-87da-574a206a3994 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:24.164900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce824-2bdf-4434-87da-574a206a3994 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:24.176174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 823/1640] 10.222.0.30 () {54 vars in 912 bytes} [Thu Jun 16 18:28:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:28:24.194376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214d3f1-f7e0-450f-a908-7d885f9a5da2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:24.194849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214d3f1-f7e0-450f-a908-7d885f9a5da2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:24.195325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214d3f1-f7e0-450f-a908-7d885f9a5da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:24.267476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5214d3f1-f7e0-450f-a908-7d885f9a5da2 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:24.269786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5214d3f1-f7e0-450f-a908-7d885f9a5da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:24.294541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 818/1641] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:24.329962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce824-2bdf-4434-87da-574a206a3994 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:24.353526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce824-2bdf-4434-87da-574a206a3994 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:24.364081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd92083-e3ce-4a61-8570-54f87037721c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:24.365204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd92083-e3ce-4a61-8570-54f87037721c service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:24.365993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd92083-e3ce-4a61-8570-54f87037721c service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:24.366636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd92083-e3ce-4a61-8570-54f87037721c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:24.385642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd92083-e3ce-4a61-8570-54f87037721c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:24.388436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd92083-e3ce-4a61-8570-54f87037721c service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:24.405814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 824/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:24.745516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f889-1575-4bff-9ed3-d82cbe8abfba None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:24.745935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f889-1575-4bff-9ed3-d82cbe8abfba None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:24.746239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708f889-1575-4bff-9ed3-d82cbe8abfba None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:24.750013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 819/1643] 10.222.0.30 () {54 vars in 912 bytes} [Thu Jun 16 18:28:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:28:24.765112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:24.767342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:24.767762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:24.833888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:24.836602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:24.926703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 825/1644] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:24.960530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708f889-1575-4bff-9ed3-d82cbe8abfba None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:25.001433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708f889-1575-4bff-9ed3-d82cbe8abfba None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:25.017302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98274c0-95a7-4f04-8394-9573c8acacd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:25.021685 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98274c0-95a7-4f04-8394-9573c8acacd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:25.022679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98274c0-95a7-4f04-8394-9573c8acacd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:25.023496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98274c0-95a7-4f04-8394-9573c8acacd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:25.086169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98274c0-95a7-4f04-8394-9573c8acacd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:25.092659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98274c0-95a7-4f04-8394-9573c8acacd6 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:25.153447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 820/1645] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:26.233432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:26.466059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:26.516557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41e86-5f88-4a12-adfb-02180c1f1486 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:26.526900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:26.529780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:26.530651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:26.531375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] PATH_INFO: `/v3/users/bc80a29a78af4201becf229b0a964fa3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:26.534982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc80a29a78af4201becf229b0a964fa3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:26.537321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:28:26.537321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 16 18:28:26.537321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: Jun 16 18:28:26.540725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:26.737928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc80a29a78af4201becf229b0a964fa3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:26.793427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc80a29a78af4201becf229b0a964fa3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:26.830987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc80a29a78af4201becf229b0a964fa3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:26.832308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc80a29a78af4201becf229b0a964fa3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:26.835476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc80a29a78af4201becf229b0a964fa3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:26.872257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 826/1646] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:26 2022] DELETE /identity/v3/users/bc80a29a78af4201becf229b0a964fa3 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:26.882148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:26.882925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:26.883695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:27.022962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:27.024942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:27.197261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 821/1647] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:27.228866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:27.234242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:27.425520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e747f4-e32c-48d4-a050-18bfae5d0b24 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:27.439040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17acd38d-2144-4e45-82f7-be63b59a98e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:27.440185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acd38d-2144-4e45-82f7-be63b59a98e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:27.440629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acd38d-2144-4e45-82f7-be63b59a98e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:27.441106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acd38d-2144-4e45-82f7-be63b59a98e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:27.476512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acd38d-2144-4e45-82f7-be63b59a98e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:27.478364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acd38d-2144-4e45-82f7-be63b59a98e6 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:27.525026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 827/1648] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:30.561577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:30.565139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:30.627063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301dd4-395b-4b26-86c4-ad46220af6fb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:30.651336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:30.655527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:30.660983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:30.661818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] PATH_INFO: `/v3/projects/a367968c80bb4ce9973d18c1674c01b1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:30.665021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a367968c80bb4ce9973d18c1674c01b1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:30.670726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:28:30.670726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 16 18:28:30.670726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: Jun 16 18:28:30.675510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:30.837280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a367968c80bb4ce9973d18c1674c01b1', 'request_id': 'req-85511c03-f7e0-482e-865c-93fcc6ce4871'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:30.850065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:30.852872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a367968c80bb4ce9973d18c1674c01b1', 'request_id': 'req-85511c03-f7e0-482e-865c-93fcc6ce4871'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:30.901732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a367968c80bb4ce9973d18c1674c01b1', 'request_id': 'req-85511c03-f7e0-482e-865c-93fcc6ce4871'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:30.957900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] The token cache is being invalidate because project a367968c80bb4ce9973d18c1674c01b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:30.958523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:30.960836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85511c03-f7e0-482e-865c-93fcc6ce4871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:30.968603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 822/1649] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:30 2022] DELETE /identity/v3/projects/a367968c80bb4ce9973d18c1674c01b1 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:30.976033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:30.976671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:30.977101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:31.067601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:31.069936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:31.414606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 828/1650] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:31.418701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:31.418934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:31.419260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:31.567325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:31.573476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:31.629850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 823/1651] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:31.630887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:31.644688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c99ee-ea16-4418-b625-bfdde74e80b2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:31.663036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:31.663752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:31.664219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:31.664501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:31.665272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:31.665583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:31.669115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:31.697280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:31.713788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:31.716681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 829/1652] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:28:31.744663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e27f21-951f-4623-8225-fc84f47498ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:31.784337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:31.786412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:31.787179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:31.788026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:31.792070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:31.797285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:31.857280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 824/1653] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:31 2022] POST /identity/v3/projects => generated 359 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:31.862115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:31.892646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f9e0f-1d7b-4223-bc9b-91a854fbbc48 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:32.196310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:32.201489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:32.202820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:32.203547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:32.205195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:32.208779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:32.373492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 830/1654] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:31 2022] POST /identity/v3/users => generated 280 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:32.375461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:32.439107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752438cf-eee5-437c-bc07-afbd3ebe106f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:32.471469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:32.472704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:32.473407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:32.478775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:32.488166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:32.533291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:32.644977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 825/1655] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:32.650276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:32.707476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da25867-4c97-41a5-ac3f-87f73969f7dc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:32.742681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41801d2-1b7e-4aa4-b5aa-a33947f12e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:32.743607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41801d2-1b7e-4aa4-b5aa-a33947f12e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:32.743898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41801d2-1b7e-4aa4-b5aa-a33947f12e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:32.744262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41801d2-1b7e-4aa4-b5aa-a33947f12e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:32.745912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41801d2-1b7e-4aa4-b5aa-a33947f12e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:32.756799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41801d2-1b7e-4aa4-b5aa-a33947f12e1e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:32.795484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 831/1656] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:32.797041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:32.820761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc58665-b34a-4723-b2d0-308b14f06af2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:32.841198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:32.842666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:32.843177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:32.843675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] PATH_INFO: `/v3/projects/c725764afce74ce284786adcd1a36498/users/15686b1f226a47f29ab60ab554a95cee/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:32.947117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c725764afce74ce284786adcd1a36498, user_id=15686b1f226a47f29ab60ab554a95cee, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:32.953350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:33.046559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 826/1657] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:32 2022] PUT /identity/v3/projects/c725764afce74ce284786adcd1a36498/users/15686b1f226a47f29ab60ab554a95cee/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:33.060565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:33.060888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:33.061126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:33.167483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] MFA Rules not processed for user `15686b1f226a47f29ab60ab554a95cee`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:33.170136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:33.364621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 832/1658] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:33 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:28:33.440145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:33.443657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:33.582283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa4fd-8c0f-4494-b207-9ed3f1d952d7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:33.603894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d154-a050-4219-a54a-86fa97277efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:33.607725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d154-a050-4219-a54a-86fa97277efb service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:33.608400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d154-a050-4219-a54a-86fa97277efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:33.608863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d154-a050-4219-a54a-86fa97277efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:33.642013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d154-a050-4219-a54a-86fa97277efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:33.643953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d154-a050-4219-a54a-86fa97277efb service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:33.684922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 827/1659] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:33.690188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:33.955945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640ec19-d9f5-4ca6-95a5-06d817d25d1b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:33.974739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:33.977750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:33.983219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:33.984436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] PATH_INFO: `/v3/users/15686b1f226a47f29ab60ab554a95cee` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:33.992329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15686b1f226a47f29ab60ab554a95cee)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:34.021313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:34.077495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:34.234795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15686b1f226a47f29ab60ab554a95cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:34.295715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15686b1f226a47f29ab60ab554a95cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:34.330309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15686b1f226a47f29ab60ab554a95cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:34.331649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15686b1f226a47f29ab60ab554a95cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:34.333097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15686b1f226a47f29ab60ab554a95cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:34.387791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 833/1660] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:33 2022] DELETE /identity/v3/users/15686b1f226a47f29ab60ab554a95cee => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:28:34.393722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:34.402678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:34.403508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:34.466981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:34.468862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:34.701740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 828/1661] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:35.072001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:35.075925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:35.265508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4468b29-081b-48ab-bd28-08c612440a60 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:35.284456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed0973-5cd5-4e95-ba5a-917cc5082a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:35.285427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed0973-5cd5-4e95-ba5a-917cc5082a45 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:35.285713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed0973-5cd5-4e95-ba5a-917cc5082a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:35.285997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed0973-5cd5-4e95-ba5a-917cc5082a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:35.331045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed0973-5cd5-4e95-ba5a-917cc5082a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:35.333145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed0973-5cd5-4e95-ba5a-917cc5082a45 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:35.366137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 834/1662] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:37.238271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:37.240283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:37.269723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b7cb-d86e-4d74-a700-ed28ee4b2030 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:37.291038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:37.292558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:37.293004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:37.293451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] PATH_INFO: `/v3/users/7a4d1e0b870f472fbd2648f0d5f45e03` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:37.295314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4d1e0b870f472fbd2648f0d5f45e03)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:37.298030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:37.373656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:37.451928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4d1e0b870f472fbd2648f0d5f45e03'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:37.468877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4d1e0b870f472fbd2648f0d5f45e03'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:37.498178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4d1e0b870f472fbd2648f0d5f45e03'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:37.536817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4d1e0b870f472fbd2648f0d5f45e03'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:37.537988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4d1e0b870f472fbd2648f0d5f45e03'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:37.554741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 829/1663] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:37 2022] DELETE /identity/v3/users/7a4d1e0b870f472fbd2648f0d5f45e03 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:37.560530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:37.561363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:37.561363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:37.595064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:37.597396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:37.756019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 835/1664] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:28:37.780861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:37.782935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:37.923038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acd0fc-971e-4819-ad30-c8388e0cbdaa admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:37.938710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b833fd-f9a4-4bfe-bab5-87a2afa57008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:37.939413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b833fd-f9a4-4bfe-bab5-87a2afa57008 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:37.939703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b833fd-f9a4-4bfe-bab5-87a2afa57008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:37.939982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b833fd-f9a4-4bfe-bab5-87a2afa57008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:37.944839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:37.946963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:37.981953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b833fd-f9a4-4bfe-bab5-87a2afa57008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:37.983495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b833fd-f9a4-4bfe-bab5-87a2afa57008 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:37.989993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbf448-9ad4-47b8-a8cb-4c6bdcddd260 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:38.005163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:38.006112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:38.006569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:38.007024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] PATH_INFO: `/v3/projects/c725764afce74ce284786adcd1a36498` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:38.012618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c725764afce74ce284786adcd1a36498)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:38.022102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:38.048866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 830/1665] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:38.104200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:38.147843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c725764afce74ce284786adcd1a36498', 'request_id': 'req-ab0f6c88-c800-4010-8c70-79b10bcc2efd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:38.160375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c725764afce74ce284786adcd1a36498', 'request_id': 'req-ab0f6c88-c800-4010-8c70-79b10bcc2efd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:38.182603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c725764afce74ce284786adcd1a36498', 'request_id': 'req-ab0f6c88-c800-4010-8c70-79b10bcc2efd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:38.219833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] The token cache is being invalidate because project c725764afce74ce284786adcd1a36498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:38.220591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:38.224907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ab0f6c88-c800-4010-8c70-79b10bcc2efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:38.229337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 836/1666] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:37 2022] DELETE /identity/v3/projects/c725764afce74ce284786adcd1a36498 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:38.243131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:38.244715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:38.245579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:38.478159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:38.482128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:38.726336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 831/1667] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:38.734776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:38.735310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:38.735310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:38.773348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:38.774793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:38.804428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 837/1668] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:38.805643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:38.822712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901601a-b6a9-4a8a-927c-f8295a7cd0bc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:38.843016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:38.844172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:38.848655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:38.849308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:38.850537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:38.851121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:38.853083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:38.936039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 832/1669] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:28:38.939082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:38.942879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:38.951224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:38.960827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab34479-f6cb-41b5-9d25-a86850949923 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:38.966980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d71ac-d4be-4071-9843-a45bf574bca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:38.967652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d71ac-d4be-4071-9843-a45bf574bca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:38.974650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d71ac-d4be-4071-9843-a45bf574bca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:38.976509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d71ac-d4be-4071-9843-a45bf574bca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:38.977774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d71ac-d4be-4071-9843-a45bf574bca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:38.983288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d71ac-d4be-4071-9843-a45bf574bca2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:39.023932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa12cb-2d89-4788-a3e0-2bdddc0372ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:39.039083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:39.043560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:39.046218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:39.046984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] PATH_INFO: `/v3/users/be1a243261e94da2af29163669af57e7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:39.049883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1a243261e94da2af29163669af57e7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:39.056337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:39.136892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 838/1670] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:38 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:39.138694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe13017-685f-4128-a757-4df9c237d0d9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:39.146534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe13017-685f-4128-a757-4df9c237d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:39.147127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe13017-685f-4128-a757-4df9c237d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:39.163471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:39.225166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bbe13017-685f-4128-a757-4df9c237d0d9 None None] MFA Rules not processed for user `89f6c755dbf74640a372802f02733eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:39.226807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bbe13017-685f-4128-a757-4df9c237d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:39.263025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1a243261e94da2af29163669af57e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:39.288325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1a243261e94da2af29163669af57e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:39.309941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1a243261e94da2af29163669af57e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:39.338628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1a243261e94da2af29163669af57e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:39.340034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1a243261e94da2af29163669af57e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:39.363774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 833/1671] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:38 2022] DELETE /identity/v3/users/be1a243261e94da2af29163669af57e7 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:39.364799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:39.367329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:39.417939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 839/1672] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:39.421596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:39.422106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:39.422534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:39.481806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:39.483255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:39.570637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24aec-c0e4-4e2f-9c99-f0c7c4572bd5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:39.586453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:39.588449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:39.589109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:39.590075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:39.592334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:39.595167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:39.674039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 834/1673] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:39 2022] POST /identity/v3/users => generated 286 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:39.675288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:39.682163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:39.700443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 840/1674] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:39.703758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:39.748836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec986dc2-21cb-41d3-ab1a-8fbdb61e6f0a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:39.781430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:39.782571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:39.783077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:39.783536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:39.790058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:39.794186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:39.858889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 841/1675] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:39.864172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:39.870362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:39.902169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:40.031269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa0dea-a12a-4695-90f7-8e704573bc70 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:40.046778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:40.047485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:40.047779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:40.048094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:40.118032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:40.191600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bafad-4c4b-48bc-8d6f-fb4f4bf31e8a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:40.218230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:40.222494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:40.223269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:40.223950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:40.256039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:40.257787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:40.286376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:40.296931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:40.303350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 835/1676] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:40.304401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:40.356160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f889-7fc6-49b3-a17b-eed7f7dff2ab service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:40.371922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 842/1677] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:40.379706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18da185-1912-4865-9712-a7ee81857e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:40.380502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da185-1912-4865-9712-a7ee81857e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:40.380795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da185-1912-4865-9712-a7ee81857e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:40.381135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da185-1912-4865-9712-a7ee81857e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:40.382145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18da185-1912-4865-9712-a7ee81857e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:40.383698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18da185-1912-4865-9712-a7ee81857e59 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:40.443811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 836/1678] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:40.450547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:40.497361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b78111-590b-437a-9a34-b7a5e8909bfb service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:40.537045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:40.539058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:40.539528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:40.540024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] PATH_INFO: `/v3/projects/40e11db7265d44d78191fbeddf7d63b2/users/bf228f7bed9d43f3b591968c5e3b30c6/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:40.633774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e11db7265d44d78191fbeddf7d63b2, user_id=bf228f7bed9d43f3b591968c5e3b30c6, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:40.642674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:40.727103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 843/1679] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:40 2022] PUT /identity/v3/projects/40e11db7265d44d78191fbeddf7d63b2/users/bf228f7bed9d43f3b591968c5e3b30c6/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:40.754127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:40.754374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:40.754587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:40.866492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] MFA Rules not processed for user `bf228f7bed9d43f3b591968c5e3b30c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:40.872829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:41.098657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:41.102681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:41.257551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 837/1680] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:41.270261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:41.272772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:41.414704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e5c0d-91e8-44b4-b62a-b2f7e3c4bb6b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:41.415888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964de0e-f103-42f5-8240-9024289ebd58 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:41.429813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:41.433836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:41.434568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:41.435020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:41.439359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:41.440614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:41.441056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:41.441480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] PATH_INFO: `/v3/projects/7bca05f0fa1b4e55b61cfa6991aceea1` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:41.444925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bca05f0fa1b4e55b61cfa6991aceea1)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:41.449623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:41.451958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:41.454096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:41.474139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 838/1681] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:41.546351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:41.576223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2e47b-f7de-46fb-a07a-2af9af8a1087 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:41.596383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:41.597404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:41.597811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:41.598097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] PATH_INFO: `/v3/users/bf228f7bed9d43f3b591968c5e3b30c6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:41.600376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf228f7bed9d43f3b591968c5e3b30c6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:41.615142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:41.671196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bca05f0fa1b4e55b61cfa6991aceea1', 'request_id': 'req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.704077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bca05f0fa1b4e55b61cfa6991aceea1', 'request_id': 'req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.724964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bca05f0fa1b4e55b61cfa6991aceea1', 'request_id': 'req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.749414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:41.767156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] The token cache is being invalidate because project 7bca05f0fa1b4e55b61cfa6991aceea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:41.767704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.767802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf228f7bed9d43f3b591968c5e3b30c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.772456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9dbbeb42-3b15-4cf9-8a2f-f5317509efb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.775519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 844/1682] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:41 2022] DELETE /identity/v3/projects/7bca05f0fa1b4e55b61cfa6991aceea1 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:28:41.781818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537d36-910c-45e7-a59b-008fbca6da7c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:41.782295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537d36-910c-45e7-a59b-008fbca6da7c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:41.782769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537d36-910c-45e7-a59b-008fbca6da7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:41.802327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf228f7bed9d43f3b591968c5e3b30c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.844278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf228f7bed9d43f3b591968c5e3b30c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.895465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fb537d36-910c-45e7-a59b-008fbca6da7c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:41.898139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fb537d36-910c-45e7-a59b-008fbca6da7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:41.948137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:41.950699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf228f7bed9d43f3b591968c5e3b30c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.952623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb2718fb-91e6-4e0d-847d-b954fe9394eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf228f7bed9d43f3b591968c5e3b30c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:41.999882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 839/1683] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:41 2022] DELETE /identity/v3/users/bf228f7bed9d43f3b591968c5e3b30c6 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:42.021187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:42.029462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:42.030287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:42.113805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:42.116072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:42.162429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 845/1684] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:42.175555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:42.176583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:42.177397 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:42.276448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:42.281614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:42.548324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 846/1685] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:28:42.551496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:42.594116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd53aa3-814f-4018-ab04-d0b97103436e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:42.624280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:42.625553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:42.626142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:42.626749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:42.628545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:42.631648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:42.642342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:42.670940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 840/1686] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 661 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:42.698092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 847/1687] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:42.699801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:42.798153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7af495-87ef-41b6-8770-8d2d5f0bdfa8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:42.823313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:42.827886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:42.848767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:42.849501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:42.853937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:42.867878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:43.038291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:43.051003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 841/1688] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:42 2022] POST /identity/v3/projects => generated 373 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:28:43.099152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92711417-14a9-4315-8c6b-0827121d61e9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:43.381613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:43.389370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:43.422936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:43.424104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:43.424578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:43.425042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:43.426336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:43.436100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:43.533765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 848/1689] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:43 2022] POST /identity/v3/users => generated 287 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:43.539961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:43.574167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb68b6-a64d-4a6f-9166-68ab9b42afed admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:43.601946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:43.603263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:43.603765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:43.604273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:43.605709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:43.608418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:43.618395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424afc7-614d-4fa4-ba38-bfab7863bc46 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:43.626922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdd0dd-1185-48ab-ab14-ff7c5d8629ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:43.628171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdd0dd-1185-48ab-ab14-ff7c5d8629ca service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:43.628636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdd0dd-1185-48ab-ab14-ff7c5d8629ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:43.629096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdd0dd-1185-48ab-ab14-ff7c5d8629ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:43.661120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 849/1690] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:43.662748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdd0dd-1185-48ab-ab14-ff7c5d8629ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:43.662904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:43.667551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdd0dd-1185-48ab-ab14-ff7c5d8629ca service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:43.708286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 842/1691] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:28:43.708931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cdaf8-f7f8-4a93-9d49-3fba4037b259 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:43.739707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:43.740997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:43.741469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:43.741971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:43.743354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:43.748950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:43.821807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 850/1692] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:43.833043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8406-8437-40d2-b6c2-ee2ba945c5b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:43.833595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8406-8437-40d2-b6c2-ee2ba945c5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:43.834070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8406-8437-40d2-b6c2-ee2ba945c5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:43.835700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:43.885865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006ddd0-b10e-4711-ba05-e84912ff8e88 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:43.911640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ecfb3-2587-4e96-954b-29d7540ea6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:43.912505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ecfb3-2587-4e96-954b-29d7540ea6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:43.912655 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ecfb3-2587-4e96-954b-29d7540ea6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:43.912916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ecfb3-2587-4e96-954b-29d7540ea6d8 admin admin] PATH_INFO: `/v3/projects/0d0cf5a8cccb491395ffceb4018300f9/users/a3072df3bc394dd6ab3dd14cd24c1d50/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:43.924125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:43.936209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-089a8406-8437-40d2-b6c2-ee2ba945c5b4 None None] MFA Rules not processed for user `73dc3ab6bc6448a89b26e3e1a1b5c6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:43.937555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-089a8406-8437-40d2-b6c2-ee2ba945c5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:43.976317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ecfb3-2587-4e96-954b-29d7540ea6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0cf5a8cccb491395ffceb4018300f9, user_id=a3072df3bc394dd6ab3dd14cd24c1d50, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:43.979321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ecfb3-2587-4e96-954b-29d7540ea6d8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:44.030718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 851/1693] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:43 2022] PUT /identity/v3/projects/0d0cf5a8cccb491395ffceb4018300f9/users/a3072df3bc394dd6ab3dd14cd24c1d50/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:44.033404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:44.033860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:44.034194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:44.054985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:44.117750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] MFA Rules not processed for user `7ee9c06c774f478db33c16b784d6b914`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:44.120429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:44.223939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 843/1694] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:44.223939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efb02-58f6-4679-aa13-9aa029210855 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:44.223939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efb02-58f6-4679-aa13-9aa029210855 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:44.223939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efb02-58f6-4679-aa13-9aa029210855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:44.376979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bf8efb02-58f6-4679-aa13-9aa029210855 None None] MFA Rules not processed for user `a3072df3bc394dd6ab3dd14cd24c1d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:44.388560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bf8efb02-58f6-4679-aa13-9aa029210855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:44.431849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 852/1695] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:44.448853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:44.452717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:44.652258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 844/1696] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:44.655361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:44.660302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:44.661006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:44.708191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] MFA Rules not processed for user `7ee9c06c774f478db33c16b784d6b914`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:44.709501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:44.729393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 845/1697] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:44.735458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:44.934766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429270aa-743e-42e8-bb14-0fe5dd879d3a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:44.943150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f640bf-c4e1-4a57-bdf3-fe2e3c0640c0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:44.954072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:44.954999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:44.955291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:44.955576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:44.956453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:44.958518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:44.978813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:44.981389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:44.988291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:44.992521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:45.032125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 846/1698] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:44 2022] POST /identity/v3/projects => generated 371 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:45.035287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:45.041823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:45.178860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:45.217138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:45.218854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:45.248947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 853/1699] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:45.249897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:45.277912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a005d-c8c5-42d7-a070-92fcf063a1de admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:45.279130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783c16-17df-4198-be8c-34a221845981 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:45.299611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:45.300348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:45.300641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:45.300921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:45.310084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:45.457983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:45.459655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:45.482486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 847/1700] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:45.483394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:45.487847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:45.576664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:45.577440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:45.577732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:45.578024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:45.578755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:45.588940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:45.673741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 854/1701] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:45 2022] POST /identity/v3/users => generated 292 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:45.676266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:45.682408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:45.718511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213fb5c-06a5-4c57-a068-b0acf9b1c5b1 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:45.728364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ff405-1992-421e-8f7c-ee4a4bd8f88b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:45.736299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:45.737757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:45.739696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:45.740349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:45.740781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] PATH_INFO: `/v3/projects/40e11db7265d44d78191fbeddf7d63b2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:45.742079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:45.747358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:45.748669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e11db7265d44d78191fbeddf7d63b2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:45.749358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] PATH_INFO: `/v3/projects/51d24550b26342efa6a0a51f02b42627` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:45.752181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d24550b26342efa6a0a51f02b42627)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:45.756632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:45.758382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:45.896288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e11db7265d44d78191fbeddf7d63b2', 'request_id': 'req-8df3375b-1f17-417e-aab2-7f065ac7f39b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.903229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d24550b26342efa6a0a51f02b42627', 'request_id': 'req-52f347c4-e97e-48ac-90c5-babc63405b37'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.912359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e11db7265d44d78191fbeddf7d63b2', 'request_id': 'req-8df3375b-1f17-417e-aab2-7f065ac7f39b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.920761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d24550b26342efa6a0a51f02b42627', 'request_id': 'req-52f347c4-e97e-48ac-90c5-babc63405b37'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.928394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e11db7265d44d78191fbeddf7d63b2', 'request_id': 'req-8df3375b-1f17-417e-aab2-7f065ac7f39b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.937480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d24550b26342efa6a0a51f02b42627', 'request_id': 'req-52f347c4-e97e-48ac-90c5-babc63405b37'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.956368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] The token cache is being invalidate because project 40e11db7265d44d78191fbeddf7d63b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:45.956894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.959571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.968322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 848/1702] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:45 2022] DELETE /identity/v3/projects/40e11db7265d44d78191fbeddf7d63b2 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:45.969385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:45.972481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] The token cache is being invalidate because project 51d24550b26342efa6a0a51f02b42627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:45.972889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.974564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52f347c4-e97e-48ac-90c5-babc63405b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:45.976889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:45.978375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 855/1703] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:45 2022] DELETE /identity/v3/projects/51d24550b26342efa6a0a51f02b42627 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:45.980724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:45.980982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:45.982429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:45.991992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:46.040181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:46.041607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:46.102283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3375b-1f17-417e-aab2-7f065ac7f39b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:46.240224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 856/1704] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:46.256735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:46.260233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:46.299864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f3c6-256e-4c32-8e75-0d91211fa9c6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:46.311215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:46.312663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49729ef-f128-4179-ac9f-ed5f95ed8c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:46.313940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49729ef-f128-4179-ac9f-ed5f95ed8c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:46.316135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49729ef-f128-4179-ac9f-ed5f95ed8c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:46.316394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:46.317795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49729ef-f128-4179-ac9f-ed5f95ed8c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:46.318159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:46.318774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49729ef-f128-4179-ac9f-ed5f95ed8c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:46.318916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] PATH_INFO: `/v3/users/f33ca4bf35634adfa5dad8a3627dedf4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:46.322386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49729ef-f128-4179-ac9f-ed5f95ed8c3e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:46.323609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33ca4bf35634adfa5dad8a3627dedf4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:46.327544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:46.374097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 849/1705] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:46.375980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575ea2-9fdb-43ec-bbf1-f75c77682723 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:46.376782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575ea2-9fdb-43ec-bbf1-f75c77682723 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:46.377080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575ea2-9fdb-43ec-bbf1-f75c77682723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:46.399913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:46.469238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-44575ea2-9fdb-43ec-bbf1-f75c77682723 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:46.472255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-44575ea2-9fdb-43ec-bbf1-f75c77682723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:46.474632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33ca4bf35634adfa5dad8a3627dedf4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:46.497867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 850/1706] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:46.504528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33ca4bf35634adfa5dad8a3627dedf4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:46.504799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:46.505103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:46.505388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:46.526018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33ca4bf35634adfa5dad8a3627dedf4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:46.527476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33ca4bf35634adfa5dad8a3627dedf4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:46.528216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33ca4bf35634adfa5dad8a3627dedf4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:46.580355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 857/1707] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:46 2022] DELETE /identity/v3/users/f33ca4bf35634adfa5dad8a3627dedf4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:46.580355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:46.589802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:46.591410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] MFA Rules not processed for user `512f01cca14b447c8f6c9eebbd828eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:46.592851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:46.824805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271257a-83f2-4ac2-be29-301a493577e6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:46.833377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f82ce-4d49-4d34-a2a9-5f578a0d0a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:46.839009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f82ce-4d49-4d34-a2a9-5f578a0d0a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:46.839009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f82ce-4d49-4d34-a2a9-5f578a0d0a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:46.839009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f82ce-4d49-4d34-a2a9-5f578a0d0a5b admin admin] PATH_INFO: `/v3/projects/7f3294df07ac4246bdfe72b90677cdf8/users/659c93b25f374a54800eec0f2b1a43d9/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:46.846201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 851/1708] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:46.848145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:46.849992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:46.876675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f82ce-4d49-4d34-a2a9-5f578a0d0a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3294df07ac4246bdfe72b90677cdf8, user_id=659c93b25f374a54800eec0f2b1a43d9, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:46.888085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f82ce-4d49-4d34-a2a9-5f578a0d0a5b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:46.911109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f4fc9-8b63-444b-84cd-ebda5d252811 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:46.917528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 858/1709] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:46 2022] PUT /identity/v3/projects/7f3294df07ac4246bdfe72b90677cdf8/users/659c93b25f374a54800eec0f2b1a43d9/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:46.920817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:46.921114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:46.921427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:46.961225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] MFA Rules not processed for user `89f6c755dbf74640a372802f02733eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:46.962233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:47.017963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:47.096333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:47.101955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.102528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.103007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.106269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:47.106902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:47.108881 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c4a02-3fa0-463c-903b-8f89dc84e852 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:47.143079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 852/1710] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:47.144957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.145769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.146178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.160976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 859/1711] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:47.161678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:47.194315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] MFA Rules not processed for user `512f01cca14b447c8f6c9eebbd828eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:47.196974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:47.209612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d1141-ebef-4092-887e-ea6735a30a0f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:47.228187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:47.233871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.234372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.236115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.242731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:47.248904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:47.298061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 860/1712] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:47.299094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:47.327640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231d97-b5e3-4d2c-be2a-6da0db3b78c7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:47.341331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748308d8-3589-4fdd-9ff8-716a447ab99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:47.342033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748308d8-3589-4fdd-9ff8-716a447ab99b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.342333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748308d8-3589-4fdd-9ff8-716a447ab99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.342802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748308d8-3589-4fdd-9ff8-716a447ab99b admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.343880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748308d8-3589-4fdd-9ff8-716a447ab99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:47.348757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748308d8-3589-4fdd-9ff8-716a447ab99b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:47.374962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:47.429506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 861/1713] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:47.431521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4d63-4e50-4e0d-bfd6-2cc03c289a84 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.431738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4d63-4e50-4e0d-bfd6-2cc03c289a84 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.431973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4d63-4e50-4e0d-bfd6-2cc03c289a84 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.434461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 862/1714] 10.222.0.30 () {54 vars in 912 bytes} [Thu Jun 16 18:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:28:47.435470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4d63-4e50-4e0d-bfd6-2cc03c289a84 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:47.468942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 853/1715] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:47.470134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:47.532259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5709c22-803c-4bf2-a60c-c17ab60df343 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:47.676375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4d63-4e50-4e0d-bfd6-2cc03c289a84 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:47.706583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5b644-cf4d-49dd-8689-3a946479cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:47.707615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b644-cf4d-49dd-8689-3a946479cf1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.707907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b644-cf4d-49dd-8689-3a946479cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.708272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b644-cf4d-49dd-8689-3a946479cf1f admin admin] PATH_INFO: `/v3/projects/7f3294df07ac4246bdfe72b90677cdf8/users/659c93b25f374a54800eec0f2b1a43d9/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.713194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b644-cf4d-49dd-8689-3a946479cf1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3294df07ac4246bdfe72b90677cdf8, user_id=659c93b25f374a54800eec0f2b1a43d9, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:47.746459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b644-cf4d-49dd-8689-3a946479cf1f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:47.760002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:47.760757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.761118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.761397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.762403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:47.772867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:47.777126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 863/1716] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:47 2022] PUT /identity/v3/projects/7f3294df07ac4246bdfe72b90677cdf8/users/659c93b25f374a54800eec0f2b1a43d9/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:47.779816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4fa3f-2770-48ab-8b5e-73075e89abad None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:47.780974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4fa3f-2770-48ab-8b5e-73075e89abad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:47.781981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4fa3f-2770-48ab-8b5e-73075e89abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:47.837446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 854/1717] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:47 2022] POST /identity/v3/users => generated 287 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:47.838505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:47.849464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:47.877183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c4c4fa3f-2770-48ab-8b5e-73075e89abad None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:47.879198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c4c4fa3f-2770-48ab-8b5e-73075e89abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:47.880968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b91e2-5eeb-41c0-a477-95f7ef8d0cca admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:48.000726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 864/1718] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:48.002829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.003461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.004016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.007968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:48.008943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.009232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.009509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] PATH_INFO: `/v3/projects/7e6ebb052f3b4a22b1ef9cf83d842c0b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.015951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6ebb052f3b4a22b1ef9cf83d842c0b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:48.018217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:48.022036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:48.093585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] MFA Rules not processed for user `659c93b25f374a54800eec0f2b1a43d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:48.096617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.227818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6ebb052f3b4a22b1ef9cf83d842c0b', 'request_id': 'req-331d4d32-eeba-444d-acac-1ee1a5c8060a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:48.256834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6ebb052f3b4a22b1ef9cf83d842c0b', 'request_id': 'req-331d4d32-eeba-444d-acac-1ee1a5c8060a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:48.274665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6ebb052f3b4a22b1ef9cf83d842c0b', 'request_id': 'req-331d4d32-eeba-444d-acac-1ee1a5c8060a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:48.330529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] The token cache is being invalidate because project 7e6ebb052f3b4a22b1ef9cf83d842c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:48.336706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:48.339638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:48.348712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 855/1719] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:47 2022] DELETE /identity/v3/projects/7e6ebb052f3b4a22b1ef9cf83d842c0b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:48.349918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:48.351856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.438122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 865/1720] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:48.442832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:48.448570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.486393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9430f-b2e4-40f3-8ec8-47918fb6f767 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:48.494591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:48.533224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d4d32-eeba-444d-acac-1ee1a5c8060a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:48.547181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b52d9-a5bb-4f54-b887-17edd79e71d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:48.548758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b52d9-a5bb-4f54-b887-17edd79e71d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.549109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b52d9-a5bb-4f54-b887-17edd79e71d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.549393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b52d9-a5bb-4f54-b887-17edd79e71d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.550156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b52d9-a5bb-4f54-b887-17edd79e71d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:48.557605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b52d9-a5bb-4f54-b887-17edd79e71d8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:48.590480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 856/1721] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:48.598824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.599306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.599769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.691718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:48.695747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:48.697112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.697565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.697997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.700799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.705155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.726705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 857/1722] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:48.732416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:48.734748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:48.737388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:48.740386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.759013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 866/1723] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:48.762151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:48.796074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143a43c-a8db-41ba-bf07-77d4df655adc service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:48.812072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad7d38-dda5-414d-b5a3-6996526d9d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:48.813063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad7d38-dda5-414d-b5a3-6996526d9d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.813504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad7d38-dda5-414d-b5a3-6996526d9d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.813934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad7d38-dda5-414d-b5a3-6996526d9d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.814865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad7d38-dda5-414d-b5a3-6996526d9d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:48.817024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad7d38-dda5-414d-b5a3-6996526d9d44 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:48.833642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 867/1724] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:48.835512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cea01-6656-4547-b113-b6d62bc6a9f4 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.836240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cea01-6656-4547-b113-b6d62bc6a9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.836692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cea01-6656-4547-b113-b6d62bc6a9f4 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.838345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 868/1725] 10.222.0.30 () {54 vars in 912 bytes} [Thu Jun 16 18:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:28:48.840745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.841572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.842102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.885915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:48.887963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:48.902875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 869/1726] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:48.904447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:48.918291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaa27c-50ca-4deb-af5d-b88ffd15db1e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:48.919539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20599d-ecda-4d6d-b0ed-0c1438d423a6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:48.925182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0a65-047d-422b-b9d9-0fdaebafa947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:48.926379 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0a65-047d-422b-b9d9-0fdaebafa947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.926974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0a65-047d-422b-b9d9-0fdaebafa947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.927421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0a65-047d-422b-b9d9-0fdaebafa947 admin admin] PATH_INFO: `/v3/projects/478dbeee61b6401c9dbb31c216ce537b/users/479bda47e66544e8bfa47dad8294c180/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.930887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:48.935148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:48.936030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:48.936429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:48.966128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae0a65-047d-422b-b9d9-0fdaebafa947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478dbeee61b6401c9dbb31c216ce537b, user_id=479bda47e66544e8bfa47dad8294c180, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:48.971200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae0a65-047d-422b-b9d9-0fdaebafa947 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:49.029743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 870/1727] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:48 2022] PUT /identity/v3/projects/478dbeee61b6401c9dbb31c216ce537b/users/479bda47e66544e8bfa47dad8294c180/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:49.036879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296c542-f3a3-4c87-b077-bd687a0bef6d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.037472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296c542-f3a3-4c87-b077-bd687a0bef6d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.037898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296c542-f3a3-4c87-b077-bd687a0bef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.062585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:49.098339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9296c542-f3a3-4c87-b077-bd687a0bef6d None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:49.101960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9296c542-f3a3-4c87-b077-bd687a0bef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:49.143574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:49.145114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:49.159820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 858/1728] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:49.160880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:49.196621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b15b2-df8e-4b1a-9a0f-18e39bd5478f service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:49.228398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 871/1729] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:28:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:49.233400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.234613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.235773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.303329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] MFA Rules not processed for user `479bda47e66544e8bfa47dad8294c180`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:49.306995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:49.339627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:49.340666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.341793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.341793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.342419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:49.342941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:49.349834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:49.362498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 859/1730] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:28:49.363077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:49.365174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:49.406276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad35f6-d853-4500-b9a2-68b64740c488 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:49.412381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:49.414220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.415731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.415731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] PATH_INFO: `/v3/users/1a2db1a1cdd34ba09b1cf42c9275fa3c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.419613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2db1a1cdd34ba09b1cf42c9275fa3c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:49.425187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:49.488206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 872/1731] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:49.489568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:49.502580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:49.520694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6a21-3e5a-4812-beaf-9ca1b8f73fe4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:49.526599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:49.528486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.529034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.529791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.544581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:49.549801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:49.552435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:49.572833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 873/1732] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:49.575678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:49.591971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23c857-1fb3-412f-8f0b-51ad901a0be9 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:49.600187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:49.601236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.601744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.602232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.603171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:49.607366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:49.617151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2db1a1cdd34ba09b1cf42c9275fa3c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:49.650584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2db1a1cdd34ba09b1cf42c9275fa3c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:49.669999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 874/1733] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:49 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:49.672976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:49.676854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2db1a1cdd34ba09b1cf42c9275fa3c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:49.710986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2db1a1cdd34ba09b1cf42c9275fa3c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:49.712924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2db1a1cdd34ba09b1cf42c9275fa3c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:49.737684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 860/1734] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:49 2022] DELETE /identity/v3/users/1a2db1a1cdd34ba09b1cf42c9275fa3c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:49.737684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:49.737684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:49.843515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:49.882289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296550d8-59f5-41cb-bfff-478fedcb817b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:49.887390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:49.894809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.895101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.895383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.897763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:49.905395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe81a-06b0-49f4-ab63-e091ef0079b6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:49.930089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:49.931790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:49.932629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:49.950079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:49.971771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:50.265893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:50.286209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:50.337379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 861/1735] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:50.338313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:50.343557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:50.389089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:50.391011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd16a1d-1684-437f-afaf-58de80bd40d5 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:50.436454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 875/1736] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 766 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:50.439896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:50.439896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:50.440310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:50.568262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:50.573355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:50.786416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a0cc3-fca4-4786-88c7-b9e31c79c6b6 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:50.804982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:50.809432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:50.810512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:50.812311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:50.813679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:50.826936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:50.919904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 876/1738] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:50.920697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 862/1738] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:50 2022] POST /identity/v3/users => generated 293 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:50.922119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:50.936617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:50.942694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:50.960849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eb855-7bec-4711-85ad-52256b93c130 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:50.976464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:50.977561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:50.979547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:50.981038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:50.983378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:50.989089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:51.049205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 863/1739] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:51.049205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:51.067566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:51.074422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d7604-5bf1-42a5-b373-c9a1b87ff21f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:51.100606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:51.112337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.113532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.114229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.126672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:51.132174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:51.189962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 864/1740] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:51.192367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:51.197222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:51.213293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903419e0-4b2f-4b89-9d28-e0c1269323a6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:51.226542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:51.228670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.229612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.229988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.283719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:51.290983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:51.330150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 877/1741] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:51.331994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:51.346927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfac0c-25dd-4af9-bf4c-0e26f395c810 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:51.385982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ed140-ae4a-49a4-84e3-238c3b655898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:51.388415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ed140-ae4a-49a4-84e3-238c3b655898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.388722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ed140-ae4a-49a4-84e3-238c3b655898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.389006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ed140-ae4a-49a4-84e3-238c3b655898 admin admin] PATH_INFO: `/v3/projects/9d3b3b8585f2455a92ef0ae947808f2e/users/74e16a73ea934614819c35935ebc6fc1/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.411837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ed140-ae4a-49a4-84e3-238c3b655898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3b3b8585f2455a92ef0ae947808f2e, user_id=74e16a73ea934614819c35935ebc6fc1, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:51.429291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ed140-ae4a-49a4-84e3-238c3b655898 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:51.462057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 878/1742] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:51 2022] PUT /identity/v3/projects/9d3b3b8585f2455a92ef0ae947808f2e/users/74e16a73ea934614819c35935ebc6fc1/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:51.473386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:51.476771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3052ef-a813-4b21-99de-1991f15496b0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.476771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3052ef-a813-4b21-99de-1991f15496b0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.476771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3052ef-a813-4b21-99de-1991f15496b0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.478719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 879/1743] 10.222.0.30 () {54 vars in 960 bytes} [Thu Jun 16 18:28:51 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:28:51.481171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096605c-a4bc-4833-a435-78cb4144a8a3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.481697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096605c-a4bc-4833-a435-78cb4144a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.482142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096605c-a4bc-4833-a435-78cb4144a8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.533456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd20df9-f918-400d-b4b4-4eb5027c0547 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:51.534839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9096605c-a4bc-4833-a435-78cb4144a8a3 None None] MFA Rules not processed for user `74e16a73ea934614819c35935ebc6fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:51.537154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9096605c-a4bc-4833-a435-78cb4144a8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:51.695872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 880/1744] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:51.697683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.697979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.699146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.722500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:51.723214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.723502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.724491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:51.725323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.725941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:51.730448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:51.833570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 881/1745] 10.222.0.30 () {58 vars in 1057 bytes} [Thu Jun 16 18:28:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:51.834626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:51.836168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:51.854149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7144a-e3ae-4c1b-b5c8-1476f3e31155 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:51.868169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:51.869832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:51.871032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:51.871624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:51.874888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:51.891513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:51.895556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:51.916439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:51.918865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:51.930195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 865/1746] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:51.931140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:51.943609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 882/1747] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:51.961477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:52.033425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae26db-4f9d-4595-b1af-6b7be7812143 service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:52.086412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04a179-20f7-4080-a0b3-2e58f00b59b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:52.086412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04a179-20f7-4080-a0b3-2e58f00b59b4 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:52.086412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04a179-20f7-4080-a0b3-2e58f00b59b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:52.086412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04a179-20f7-4080-a0b3-2e58f00b59b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:52.114463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04a179-20f7-4080-a0b3-2e58f00b59b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:52.116461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04a179-20f7-4080-a0b3-2e58f00b59b4 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:52.134936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28add57-b6c5-4452-b75a-fa33d3d0314e service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:52.146557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 883/1748] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:52.154388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:52.155650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:52.157774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:52.176753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:52.177853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:52.178150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:52.178528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:52.183013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:52.184869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:52.202873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:52.254042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] MFA Rules not processed for user `73dc3ab6bc6448a89b26e3e1a1b5c6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:52.256885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:52.306737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 866/1749] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:51 2022] POST /identity/v3/projects => generated 387 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:52.312925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:52.318773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:52.395128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:52.515955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 884/1750] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:52.517118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:52.560673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864d97e-bfb1-4638-b60b-a12bd623c502 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:52.568149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:52.569462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:52.570161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:52.570839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] PATH_INFO: `/v3/projects/a9353f1b4e36450eb953f2171c4dab8a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:52.576154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9353f1b4e36450eb953f2171c4dab8a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:52.578633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:52.593547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ffa97-2e25-424b-9036-eb3574ca9d79 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:52.599537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:52.600273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:52.600563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:52.600844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:52.601562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:52.603605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:52.689286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9353f1b4e36450eb953f2171c4dab8a', 'request_id': 'req-772d9441-d3c3-4fa0-b831-a261d322ed4b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:52.699634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9353f1b4e36450eb953f2171c4dab8a', 'request_id': 'req-772d9441-d3c3-4fa0-b831-a261d322ed4b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:52.709726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9353f1b4e36450eb953f2171c4dab8a', 'request_id': 'req-772d9441-d3c3-4fa0-b831-a261d322ed4b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:52.713202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 885/1751] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:52 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:52.718615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:52.772198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207da3a-dcf2-43ca-af94-0591b895f9e4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:52.808823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] The token cache is being invalidate because project a9353f1b4e36450eb953f2171c4dab8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:52.809606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:52.839268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-772d9441-d3c3-4fa0-b831-a261d322ed4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:52.853681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 867/1752] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:52 2022] DELETE /identity/v3/projects/a9353f1b4e36450eb953f2171c4dab8a => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:52.867368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:52.867883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:52.868430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:53.008917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:53.009798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:53.010266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:53.010663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:53.011747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:53.015659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:53.060383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] MFA Rules not processed for user `479bda47e66544e8bfa47dad8294c180`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:53.067775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:53.069069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 886/1753] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:53.070457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:53.082978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:53.142729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729cd5d-59f1-4c72-90eb-5f0672e0441d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:53.174727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:53.176083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:53.176469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:53.176855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] PATH_INFO: `/v3/users/af98332fe9fa4c1d9055709561939919` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:53.184847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af98332fe9fa4c1d9055709561939919)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:53.187131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:53.360267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af98332fe9fa4c1d9055709561939919'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:53.429315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af98332fe9fa4c1d9055709561939919'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:53.468124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af98332fe9fa4c1d9055709561939919'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:53.479408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af98332fe9fa4c1d9055709561939919'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:53.481610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af98332fe9fa4c1d9055709561939919'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:53.518847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:53.558530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 887/1754] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:53 2022] DELETE /identity/v3/users/af98332fe9fa4c1d9055709561939919 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:53.562467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:53.570924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:53.732231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 868/1755] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 876 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:54.005442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9687ee-da26-4760-8413-3c5847fc6899 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:54.017713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:54.019640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:54.020426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:54.020997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] PATH_INFO: `/v3/projects/cfdc6e8a6242491aa8cf08f0f4f8cd6d/users/f46235ad6499422f9ad461ab3174abb9/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:54.079343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdc6e8a6242491aa8cf08f0f4f8cd6d, user_id=f46235ad6499422f9ad461ab3174abb9, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:54.090821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:54.138167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 888/1756] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:53 2022] PUT /identity/v3/projects/cfdc6e8a6242491aa8cf08f0f4f8cd6d/users/f46235ad6499422f9ad461ab3174abb9/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:54.139637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:54.186313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9746feb-9d6f-48f8-adcc-57f54ae4a477 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:54.467353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:54.502687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465b319-cbee-4954-a5f7-a2050b43eb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:54.504553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465b319-cbee-4954-a5f7-a2050b43eb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:54.506150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465b319-cbee-4954-a5f7-a2050b43eb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:54.513090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465b319-cbee-4954-a5f7-a2050b43eb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:54.516858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465b319-cbee-4954-a5f7-a2050b43eb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:54.524216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465b319-cbee-4954-a5f7-a2050b43eb31 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:54.568215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:54.569735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 869/1757] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:54.608372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777d303-ba38-4e8e-bdfe-b788630f9c07 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:54.642348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:54.643291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:54.643612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:54.643893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] PATH_INFO: `/v3/projects/cfdc6e8a6242491aa8cf08f0f4f8cd6d/users/f46235ad6499422f9ad461ab3174abb9/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:54.649110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdc6e8a6242491aa8cf08f0f4f8cd6d, user_id=f46235ad6499422f9ad461ab3174abb9, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:54.656907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:54.699715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 889/1758] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:54 2022] PUT /identity/v3/projects/cfdc6e8a6242491aa8cf08f0f4f8cd6d/users/f46235ad6499422f9ad461ab3174abb9/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:54.728781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:54.729663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:54.730626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:54.870875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] MFA Rules not processed for user `f46235ad6499422f9ad461ab3174abb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:54.873929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:55.163829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 870/1759] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:55.193643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:55.200307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:55.563874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ce4a-0747-4330-bdf1-aa61f03a6e35 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:55.576792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:55.577815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:55.578420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:55.579081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:55.633937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:55.642048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:55.683815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 890/1760] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:28:55.768205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:55.770699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:56.051933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:56.215291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d3bcb-3cde-4d7a-b438-4858a122319b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:56.255587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:56.265123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:56.266646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:56.273303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] PATH_INFO: `/v3/projects/133bf8c4b0684602b2b7b1f9549f17bb` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:56.275704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=133bf8c4b0684602b2b7b1f9549f17bb)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:56.281387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:56.455280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb541a-f45a-4a1c-8b7e-503045531e00 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:56.491926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:56.499571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:56.508626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:56.509808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] PATH_INFO: `/v3/users/74e16a73ea934614819c35935ebc6fc1` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:56.514502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e16a73ea934614819c35935ebc6fc1)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:56.516739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:56.586796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133bf8c4b0684602b2b7b1f9549f17bb', 'request_id': 'req-5e158818-85d5-48a9-9c47-43be636620e0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.593156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:56.612585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133bf8c4b0684602b2b7b1f9549f17bb', 'request_id': 'req-5e158818-85d5-48a9-9c47-43be636620e0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.631565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133bf8c4b0684602b2b7b1f9549f17bb', 'request_id': 'req-5e158818-85d5-48a9-9c47-43be636620e0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.673863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] The token cache is being invalidate because project 133bf8c4b0684602b2b7b1f9549f17bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:56.682100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.684321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5e158818-85d5-48a9-9c47-43be636620e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.701714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 871/1761] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:55 2022] DELETE /identity/v3/projects/133bf8c4b0684602b2b7b1f9549f17bb => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:56.707469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3de2cd-3395-48cc-ba08-29a9a427d4dc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:56.708235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e16a73ea934614819c35935ebc6fc1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.727844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3de2cd-3395-48cc-ba08-29a9a427d4dc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:56.728168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3de2cd-3395-48cc-ba08-29a9a427d4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:56.743742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:56.749185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e16a73ea934614819c35935ebc6fc1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.778600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e16a73ea934614819c35935ebc6fc1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.780697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e16a73ea934614819c35935ebc6fc1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.783353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6be6be73-23a8-4524-94cd-554340d2083d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e16a73ea934614819c35935ebc6fc1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:56.802308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8e3de2cd-3395-48cc-ba08-29a9a427d4dc None None] MFA Rules not processed for user `512f01cca14b447c8f6c9eebbd828eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:56.805702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8e3de2cd-3395-48cc-ba08-29a9a427d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:56.827542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 891/1762] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:56 2022] DELETE /identity/v3/users/74e16a73ea934614819c35935ebc6fc1 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:56.831610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210246c0-9f32-4a8e-8593-36c3f940573d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:56.832343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210246c0-9f32-4a8e-8593-36c3f940573d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:56.833670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210246c0-9f32-4a8e-8593-36c3f940573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:56.903929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-210246c0-9f32-4a8e-8593-36c3f940573d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:56.906472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-210246c0-9f32-4a8e-8593-36c3f940573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:57.096902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 872/1763] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:57.099353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.105079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.105620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.177458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 892/1764] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:57.187295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.187768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.188174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.197899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:57.203037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:57.248340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:57.256700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:57.258391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 873/1765] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:57.273024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:57.275758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:57.311093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 893/1766] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:57.316707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:57.335678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305185a-bb31-43fb-880a-c5cb7a352a8f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:57.358288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:57.359176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.363120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.363541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.365630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:28:57.366325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:57.369620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:57.409904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 894/1767] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:28:57.415633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:57.447682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82a771-43a8-4ac7-a475-f09e40c3c9fb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:57.466339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:57.467428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.467428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.467947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.468818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:57.471247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:57.535480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 895/1768] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:57 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:57.540389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:57.663824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c92fa-e16e-48c7-a2c9-3dc3518970e5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:57.692805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:57.721020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e5e23-7c0a-43e0-b320-5a7784f4a3ef admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:57.739367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:57.740273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.740674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.741059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.741905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:57.745226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:57.788635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 896/1769] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:57 2022] POST /identity/v3/users => generated 283 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:28:57.793248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:57.809322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37a80-c01a-4916-8275-4079373fdbeb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:57.819640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:57.820852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.821611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.823677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.824840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:57.828510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:57.857663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 897/1770] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:28:57.862457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:57.893172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de8120-d395-471e-b33c-72471e33e661 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:57.908641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:57.909507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:57.909906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:57.910296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:57.911298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:57.913195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:57.946801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 898/1771] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:57.958636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:58.002360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a48574-0a3e-4b7a-891a-0b427e626e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:58.008209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29155153-d944-4998-b09e-b3f4efc680fe admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:58.013198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a48574-0a3e-4b7a-891a-0b427e626e8b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:58.014485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a48574-0a3e-4b7a-891a-0b427e626e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:58.015744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a48574-0a3e-4b7a-891a-0b427e626e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:58.032721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:58.034534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:58.039102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:58.039786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] PATH_INFO: `/v3/projects/99f403f66be44d37ad1346dcb72205f2/users/6168b8cb5c8a4ec8aeace2abf828eb94/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:58.079756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a48574-0a3e-4b7a-891a-0b427e626e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:58.085148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a48574-0a3e-4b7a-891a-0b427e626e8b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:58.110467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f403f66be44d37ad1346dcb72205f2, user_id=6168b8cb5c8a4ec8aeace2abf828eb94, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:58.115986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:58.129666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 874/1772] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 858 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:28:58.184037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 899/1773] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:57 2022] PUT /identity/v3/projects/99f403f66be44d37ad1346dcb72205f2/users/6168b8cb5c8a4ec8aeace2abf828eb94/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:58.206891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:58.207408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:58.207969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:58.296697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] MFA Rules not processed for user `6168b8cb5c8a4ec8aeace2abf828eb94`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:28:58.300774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:58.552501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:58.643891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 875/1774] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:28:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:28:58.649341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:58.859779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b50a8-8958-4875-a57b-da80a4fdc22d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:58.869465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:58.871925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:58.873553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:58.874059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:58.875418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:58.879493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:58.941895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 900/1775] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:28:58 2022] POST /identity/v3/projects => generated 361 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:28:58.950254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:59.002851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:59.005121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc48f8-6967-4a33-bc8c-2ea3fe5c52c0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:59.009743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:59.273950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fe883-df1b-4dee-a499-30e89d733026 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:59.282127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:59.283782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:59.284332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:59.284826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] PATH_INFO: `/v3/projects/9d3b3b8585f2455a92ef0ae947808f2e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:59.295106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3b3b8585f2455a92ef0ae947808f2e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:59.298776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:59.302746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:59.303909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:59.304428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:59.304969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:59.306212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:59.318269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:59.360371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 876/1776] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:28:58 2022] POST /identity/v3/users => generated 287 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:28:59.366048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:59.373812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:59.396162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c34d7e-11c6-4be4-8542-240fe89f9133 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:59.407867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:59.409224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:59.409850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:59.410448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:59.412075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:59.416126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:59.449063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 877/1777] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:28:59.456793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:59.464873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3b3b8585f2455a92ef0ae947808f2e', 'request_id': 'req-50ef12e0-ba5d-4b52-a645-59a3e2409208'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:59.474510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b2107-45d0-4dee-915c-f100b0ac086e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:59.481118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3b3b8585f2455a92ef0ae947808f2e', 'request_id': 'req-50ef12e0-ba5d-4b52-a645-59a3e2409208'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:59.498646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3b3b8585f2455a92ef0ae947808f2e', 'request_id': 'req-50ef12e0-ba5d-4b52-a645-59a3e2409208'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:59.564559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] The token cache is being invalidate because project 9d3b3b8585f2455a92ef0ae947808f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:28:59.565174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:59.568675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:28:59.587262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 901/1778] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:28:58 2022] DELETE /identity/v3/projects/9d3b3b8585f2455a92ef0ae947808f2e => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:59.605563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:59.617648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:59.657677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:59.658872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:59.659323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:59.659760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] PATH_INFO: `/v3/projects/c2f57956e6694b29ad3261f6e8b233a7/users/0bc7907de6ee49e588d4dc5df210fa94/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:59.706291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f57956e6694b29ad3261f6e8b233a7, user_id=0bc7907de6ee49e588d4dc5df210fa94, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:59.709289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:59.723671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef12e0-ba5d-4b52-a645-59a3e2409208 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:28:59.740880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 878/1779] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:28:59 2022] PUT /identity/v3/projects/c2f57956e6694b29ad3261f6e8b233a7/users/0bc7907de6ee49e588d4dc5df210fa94/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:28:59.741242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:28:59.743555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:28:59.744099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:28:59.744934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] PATH_INFO: `/v3/users/f46235ad6499422f9ad461ab3174abb9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:28:59.747192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:28:59.748449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f46235ad6499422f9ad461ab3174abb9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:28:59.749067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:28:59.768719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:28:59.989570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:28:59.990712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f46235ad6499422f9ad461ab3174abb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:00.010022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f46235ad6499422f9ad461ab3174abb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:00.021681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f46235ad6499422f9ad461ab3174abb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:00.022514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f46235ad6499422f9ad461ab3174abb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:00.023422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f46235ad6499422f9ad461ab3174abb9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:00.046613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094e006-e2d9-4094-8c16-25f6d51f5fd2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:00.080937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a07fd-6178-43f6-8808-32343b6735f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:00.081698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a07fd-6178-43f6-8808-32343b6735f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:00.081984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a07fd-6178-43f6-8808-32343b6735f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:00.082265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a07fd-6178-43f6-8808-32343b6735f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:00.083052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a07fd-6178-43f6-8808-32343b6735f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:00.089477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a07fd-6178-43f6-8808-32343b6735f1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:00.093400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 902/1780] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:28:59 2022] DELETE /identity/v3/users/f46235ad6499422f9ad461ab3174abb9 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:00.140521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 879/1781] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:00.142797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:00.145326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:00.577521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46d939-2dbf-47cb-be17-c4723a7bebd4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:00.602427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:00.608427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:00.608427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:00.608427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] PATH_INFO: `/v3/projects/c2f57956e6694b29ad3261f6e8b233a7/users/0bc7907de6ee49e588d4dc5df210fa94/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:00.617308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f57956e6694b29ad3261f6e8b233a7, user_id=0bc7907de6ee49e588d4dc5df210fa94, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:00.624018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:00.669057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 903/1782] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:00 2022] PUT /identity/v3/projects/c2f57956e6694b29ad3261f6e8b233a7/users/0bc7907de6ee49e588d4dc5df210fa94/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:00.694428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:00.695349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:00.695799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:00.806800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] MFA Rules not processed for user `0bc7907de6ee49e588d4dc5df210fa94`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:00.811158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:01.113830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 880/1783] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:00 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:01.129602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:01.131488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:01.229144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:01.412208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230266f2-f5d2-404f-ad30-18c6f42e0657 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:01.439683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65543ce4-72b2-4134-9481-4c9bbe1faf0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:01.443329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65543ce4-72b2-4134-9481-4c9bbe1faf0c service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:01.444140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65543ce4-72b2-4134-9481-4c9bbe1faf0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:01.444658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65543ce4-72b2-4134-9481-4c9bbe1faf0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:01.473247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65543ce4-72b2-4134-9481-4c9bbe1faf0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:01.492420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65543ce4-72b2-4134-9481-4c9bbe1faf0c service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:01.538838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 904/1784] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:01 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:01.706682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:01.729007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87790c8-a8bf-4b1d-bafd-ee0fafbcc93a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:01.733455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:01.734252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:01.734540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:01.734833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:01.739599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:01.989003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:01.996104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:02.007027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:02.013593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:02.016185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:02.080886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 881/1785] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:02.157082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:02.166385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:02.573807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 905/1786] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:29:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:02.957840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:02.959217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:03.219348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b09c-3816-42fc-b44f-58726e9296a5 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:03.238764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9cc4f-f39b-4ecb-ada4-3020c0905740 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:03.244417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9cc4f-f39b-4ecb-ada4-3020c0905740 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:03.244954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9cc4f-f39b-4ecb-ada4-3020c0905740 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:03.246881 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9cc4f-f39b-4ecb-ada4-3020c0905740 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:03.267510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9cc4f-f39b-4ecb-ada4-3020c0905740 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:03.279127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9cc4f-f39b-4ecb-ada4-3020c0905740 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:03.297448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 882/1787] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:29:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:29:05.021528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:05.023341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:05.297674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b380a-e80c-4b4c-95e8-1905f2ad7dcc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:05.316857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:05.318122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:05.318470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:05.318948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] PATH_INFO: `/v3/projects/cfdc6e8a6242491aa8cf08f0f4f8cd6d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:05.320540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfdc6e8a6242491aa8cf08f0f4f8cd6d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:05.322999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:05.476860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfdc6e8a6242491aa8cf08f0f4f8cd6d', 'request_id': 'req-f2771d65-3181-40c9-bc9b-7517ff0b2181'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:05.522471 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfdc6e8a6242491aa8cf08f0f4f8cd6d', 'request_id': 'req-f2771d65-3181-40c9-bc9b-7517ff0b2181'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:05.535901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfdc6e8a6242491aa8cf08f0f4f8cd6d', 'request_id': 'req-f2771d65-3181-40c9-bc9b-7517ff0b2181'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:05.546969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:05.574658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] The token cache is being invalidate because project cfdc6e8a6242491aa8cf08f0f4f8cd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:05.574969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:05.577726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f2771d65-3181-40c9-bc9b-7517ff0b2181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:05.580490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 906/1788] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:05 2022] DELETE /identity/v3/projects/cfdc6e8a6242491aa8cf08f0f4f8cd6d => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:05.588470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:05.589521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:05.590026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:05.643317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:05.706103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:05.714582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:05.965358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 883/1789] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:05.977335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:05.977671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:05.978020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.034175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:06.038814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:06.065904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 907/1790] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:06.071606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:06.085952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dad082-3ddb-46e5-bc19-0b1cf0e7e56c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:06.105936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:06.114137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:06.114582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:06.115079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.118305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:06.118749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:06.122179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:06.179347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 884/1791] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:29:06.187210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:06.207126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f16bd-06a4-41cf-8280-fb25e89f6dd1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:06.224012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:06.224744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:06.225034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:06.225311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.226032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:06.228272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:06.318908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 908/1792] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:06 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:06.326355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:06.348999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271e853-13f2-430d-b5cf-37027e8e3100 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:06.583313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fbac8-aafe-44cd-b0dc-e5bef4bdb37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:06.584826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fbac8-aafe-44cd-b0dc-e5bef4bdb37a admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:06.585325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fbac8-aafe-44cd-b0dc-e5bef4bdb37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:06.585770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fbac8-aafe-44cd-b0dc-e5bef4bdb37a admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.586681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fbac8-aafe-44cd-b0dc-e5bef4bdb37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:06.589500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fbac8-aafe-44cd-b0dc-e5bef4bdb37a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:06.680819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:06.702431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 885/1793] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:06 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:06.711305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57380b-5eaf-43b4-a81b-f323f79aae63 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:06.736403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:06.739855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:06.740247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:06.746954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.748262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:06.754799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:06.781186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:06.781536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:06.781829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.848653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 909/1794] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:06.849584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:06.874876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:06.883135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:06.920940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34ed93-0b96-4ea1-9839-6c7a5bfc78cb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:06.968307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:06.969368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:06.983204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:06.988109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:06.989423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:07.015175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:07.117819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 910/1795] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:07.121363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:07.172428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc27f4-df7a-4ccc-81e1-368655113399 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:07.215606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca580dbb-4d39-499f-a2aa-1c40ca044326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:07.221706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca580dbb-4d39-499f-a2aa-1c40ca044326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:07.223004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca580dbb-4d39-499f-a2aa-1c40ca044326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:07.227850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca580dbb-4d39-499f-a2aa-1c40ca044326 admin admin] PATH_INFO: `/v3/projects/ae54f47737a64a129a8f63bdd666d945/users/36e17c6ceacc4ec0bc543b34fde28053/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:07.286942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 886/1796] 10.222.0.88 () {58 vars in 1015 bytes} [Thu Jun 16 18:29:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:07.289259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:07.292457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:07.328225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca580dbb-4d39-499f-a2aa-1c40ca044326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae54f47737a64a129a8f63bdd666d945, user_id=36e17c6ceacc4ec0bc543b34fde28053, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:07.330308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1e0b3-b68b-4e60-a47a-fca449e5bec3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:07.336633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca580dbb-4d39-499f-a2aa-1c40ca044326 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:07.362013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:07.363908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:07.370928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:07.371607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] PATH_INFO: `/v3/users/6168b8cb5c8a4ec8aeace2abf828eb94` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:07.380504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6168b8cb5c8a4ec8aeace2abf828eb94)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:07.395187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:07.428904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 911/1797] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:07 2022] PUT /identity/v3/projects/ae54f47737a64a129a8f63bdd666d945/users/36e17c6ceacc4ec0bc543b34fde28053/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:07.437796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:07.438928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:07.439732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:07.533827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] MFA Rules not processed for user `36e17c6ceacc4ec0bc543b34fde28053`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:07.537849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:07.538588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:07.668889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6168b8cb5c8a4ec8aeace2abf828eb94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:07.715681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6168b8cb5c8a4ec8aeace2abf828eb94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:07.751967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6168b8cb5c8a4ec8aeace2abf828eb94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:07.817995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6168b8cb5c8a4ec8aeace2abf828eb94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:07.819919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6168b8cb5c8a4ec8aeace2abf828eb94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:07.852359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 887/1798] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:07 2022] DELETE /identity/v3/users/6168b8cb5c8a4ec8aeace2abf828eb94 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:07.860987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:07.869154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:07.944485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 912/1799] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:07.954007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:07.955727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:08.107505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2420a-961b-479b-9385-1bb2d2cd6e11 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:08.115557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:08.118328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:08.118630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:08.118775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:08.125472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:08.150580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:08.156273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bd3f0-984e-450f-9d3c-a50f921d99ea service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:08.162328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9644-7dab-486a-824e-d5f8755b4f36 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:08.172776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:08.174805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:08.175361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:08.175937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:08.189614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 888/1800] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:08.193683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:08.196225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:08.197071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:08.202479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:08.212946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:08.253937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:08.258611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 913/1801] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:08.261213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:08.264274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:08.273745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:08.373382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 889/1802] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:08.374411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:08.541433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:08.544356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:08.591526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d76936-b18e-4194-b84b-0bad51bd16ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:08.600603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:08.601580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:08.602096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:08.603813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:08.620782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:08.624587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:08.649051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 890/1803] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:08.650164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400598-b694-437a-b638-fa87ce90e8a3 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:08.681112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:08.682891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:08.683398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:08.683903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:08.693712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:08.716852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:08.839141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 914/1804] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:08 2022] POST /identity/v3/projects => generated 387 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:08.842999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:08.862350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed135-1f5c-4578-9e85-e0dd73bb710e service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:08.983157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:09.099280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:09.100693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:09.101405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:09.101894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:09.103273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:09.124775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:09.224777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:09.225823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 891/1805] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:08 2022] POST /identity/v3/users => generated 294 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:09.277679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c75535-1665-4b01-bbd9-a177e1f4ecf6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:09.299616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:09.300807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:09.301651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:09.302364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:09.304180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:09.308098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:09.332513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 915/1806] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:09.338491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:09.374664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def893b-649d-476a-8b3b-8d3b645cfe77 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:09.388883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238329d-13a4-4802-917c-7ed404725b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:09.390101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238329d-13a4-4802-917c-7ed404725b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:09.390515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238329d-13a4-4802-917c-7ed404725b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:09.390990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238329d-13a4-4802-917c-7ed404725b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:09.392434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238329d-13a4-4802-917c-7ed404725b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:09.395962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238329d-13a4-4802-917c-7ed404725b7f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:09.443804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:09.458950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 892/1807] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:09.465657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d518bd-2a40-4800-bb5c-4179e37e21e7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:09.481106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:09.482012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:09.482294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:09.482599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] PATH_INFO: `/v3/projects/18602dd042284868a6cd8fcfd1116815/users/0c9b9f0b79f54cdcacc6ea042845c3b1/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:09.520550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18602dd042284868a6cd8fcfd1116815, user_id=0c9b9f0b79f54cdcacc6ea042845c3b1, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:09.523681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:09.561926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 916/1808] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:09 2022] PUT /identity/v3/projects/18602dd042284868a6cd8fcfd1116815/users/0c9b9f0b79f54cdcacc6ea042845c3b1/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:09.579800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:09.580429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:09.580989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:09.713649 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] MFA Rules not processed for user `0c9b9f0b79f54cdcacc6ea042845c3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:09.726523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:10.088097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 893/1809] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:10.093484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:10.392551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974d59-e77c-4061-ac63-214029e7fba6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:10.416036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:10.418984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:10.419855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:10.434614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:10.440093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:10.449981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:10.605066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:10.605440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 917/1810] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:10 2022] POST /identity/v3/projects => generated 385 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:10.640535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa612cd-ceb8-4f72-905c-aca39367d846 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:10.876524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:10.877231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:10.877518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:10.877797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:10.879129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:10.880601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:10.928825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 894/1811] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:10 2022] POST /identity/v3/users => generated 299 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:10.937767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:10.971901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221cac-cd21-453c-8776-08a8089c6b94 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:10.981654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:10.982979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:10.988479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:10.994476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:10.998522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:11.044976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:11.110799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:11.121833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 918/1812] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:11.162184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd29fe-9e05-431a-8d24-28cf35d3ee74 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:11.177740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:11.178918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:11.178918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:11.182724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] PATH_INFO: `/v3/projects/5fa2f314b54749e9b2e397ffe5bd6683/users/7b5a77e267084f18918aa8c11d1495a4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:11.234543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa2f314b54749e9b2e397ffe5bd6683, user_id=7b5a77e267084f18918aa8c11d1495a4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:11.241667 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:11.334542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 895/1813] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:11 2022] PUT /identity/v3/projects/5fa2f314b54749e9b2e397ffe5bd6683/users/7b5a77e267084f18918aa8c11d1495a4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:11.336979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:11.368536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522248d-7c5b-4aff-b19e-032609e228bc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:11.523192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:11.674228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5446d-60a5-4dc5-afa4-c0e773ab0e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:11.686539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5446d-60a5-4dc5-afa4-c0e773ab0e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:11.687826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5446d-60a5-4dc5-afa4-c0e773ab0e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:11.688950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5446d-60a5-4dc5-afa4-c0e773ab0e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:11.696131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5446d-60a5-4dc5-afa4-c0e773ab0e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:11.707417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5446d-60a5-4dc5-afa4-c0e773ab0e72 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:11.776708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 919/1814] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:11.783820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:11.808129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:11.824198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e364a27-c4bd-4d56-874a-1502aa4f9b34 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:11.849258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:11.851305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:11.851910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:11.853904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] PATH_INFO: `/v3/projects/5fa2f314b54749e9b2e397ffe5bd6683/users/7b5a77e267084f18918aa8c11d1495a4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:11.866600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa2f314b54749e9b2e397ffe5bd6683, user_id=7b5a77e267084f18918aa8c11d1495a4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:11.875010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:11.925696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 896/1815] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:11 2022] PUT /identity/v3/projects/5fa2f314b54749e9b2e397ffe5bd6683/users/7b5a77e267084f18918aa8c11d1495a4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:11.952835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:11.953352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:11.953911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:12.075540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] MFA Rules not processed for user `7b5a77e267084f18918aa8c11d1495a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:12.091540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:12.351210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 920/1816] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:12.368068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:12.371218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:12.568767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37be7a-aae5-4ac3-8308-f671c7708453 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:12.591096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:12.595628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:12.597916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:12.599404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:12.605426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:12.607714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:12.656659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:12.664333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:12.721011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 897/1817] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:13.215219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfd40d-5fde-411d-aeff-f793bfc2dd25 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:13.230977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:13.232326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:13.232832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:13.233332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] PATH_INFO: `/v3/projects/99f403f66be44d37ad1346dcb72205f2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:13.236359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f403f66be44d37ad1346dcb72205f2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:13.243467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:13.455084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f403f66be44d37ad1346dcb72205f2', 'request_id': 'req-6701b953-9394-414c-b760-c0edb41c4d68'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:13.486866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f403f66be44d37ad1346dcb72205f2', 'request_id': 'req-6701b953-9394-414c-b760-c0edb41c4d68'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:13.526936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:13.533899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f403f66be44d37ad1346dcb72205f2', 'request_id': 'req-6701b953-9394-414c-b760-c0edb41c4d68'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:13.583504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] The token cache is being invalidate because project 99f403f66be44d37ad1346dcb72205f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:13.584280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:13.588490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6701b953-9394-414c-b760-c0edb41c4d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:13.600322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:13.614339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 921/1818] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:12 2022] DELETE /identity/v3/projects/99f403f66be44d37ad1346dcb72205f2 => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:13.627481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f995a-09f4-494b-bda7-b808848249e3 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:13.909893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:13.910823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:13.911489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:13.911489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] PATH_INFO: `/v3/users/0bc7907de6ee49e588d4dc5df210fa94` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:13.916289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc7907de6ee49e588d4dc5df210fa94)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:13.919406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:14.058545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc7907de6ee49e588d4dc5df210fa94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:14.093352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc7907de6ee49e588d4dc5df210fa94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:14.143638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc7907de6ee49e588d4dc5df210fa94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:14.256367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc7907de6ee49e588d4dc5df210fa94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:14.275315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc7907de6ee49e588d4dc5df210fa94'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:14.352141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 898/1819] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:13 2022] DELETE /identity/v3/users/0bc7907de6ee49e588d4dc5df210fa94 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:14.761841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0ffe-f1b5-4627-b96e-17c5c4224586 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:14.762470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0ffe-f1b5-4627-b96e-17c5c4224586 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:14.762470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0ffe-f1b5-4627-b96e-17c5c4224586 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:14.763858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 922/1820] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:29:14.892720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:14.895830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:14.919857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:14.921023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:14.922178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:14.994141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:15.070156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:15.071980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:15.191872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869572d-70d5-4006-a33b-d556883e957d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:15.227819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:15.228670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:15.231629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:15.232224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:15.241053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:15.344033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 923/1821] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:29:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:15.517265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:15.519849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683fcc3-4686-4349-8666-b3dbf3b30813 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:15.550857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 899/1822] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:17.924117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:17.925695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:17.942497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5820b-6d87-4d56-9280-975b28631418 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:17.949076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a561e2-e24f-44de-a968-26e6065bc3b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:17.949988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a561e2-e24f-44de-a968-26e6065bc3b3 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:17.950279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a561e2-e24f-44de-a968-26e6065bc3b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:17.950561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a561e2-e24f-44de-a968-26e6065bc3b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:17.961403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a561e2-e24f-44de-a968-26e6065bc3b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:17.963055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a561e2-e24f-44de-a968-26e6065bc3b3 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:17.988503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 924/1823] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:18.659375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231294-41fd-446f-b40f-7ed07a01554e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:18.660480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231294-41fd-446f-b40f-7ed07a01554e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:18.660913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231294-41fd-446f-b40f-7ed07a01554e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:18.662937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 900/1824] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:29:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:29:18.730741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:18.731743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:18.732484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:18.790698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:18.795228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:18.814981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 925/1825] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:29:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:18.840539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231294-41fd-446f-b40f-7ed07a01554e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:18.843078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21231294-41fd-446f-b40f-7ed07a01554e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:19.072887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231294-41fd-446f-b40f-7ed07a01554e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:19.084942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:19.089055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:19.089055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:19.089055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] PATH_INFO: `/v3/projects/c2f57956e6694b29ad3261f6e8b233a7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:19.090636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f57956e6694b29ad3261f6e8b233a7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:19.095744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:19.190911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:19.194126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:19.272280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f57956e6694b29ad3261f6e8b233a7', 'request_id': 'req-d043d36f-4787-4dc5-a7d1-00068566e168'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:19.301135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f57956e6694b29ad3261f6e8b233a7', 'request_id': 'req-d043d36f-4787-4dc5-a7d1-00068566e168'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:19.315632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fb7b2-c72b-43dc-8c24-28e823c03e4e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:19.318442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f57956e6694b29ad3261f6e8b233a7', 'request_id': 'req-d043d36f-4787-4dc5-a7d1-00068566e168'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:19.347426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:19.356470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] The token cache is being invalidate because project c2f57956e6694b29ad3261f6e8b233a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:19.356944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:19.361364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d043d36f-4787-4dc5-a7d1-00068566e168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:19.369359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 901/1826] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:18 2022] DELETE /identity/v3/projects/c2f57956e6694b29ad3261f6e8b233a7 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:19.376373 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a4325-3ea7-42f3-853d-76da6c417bde None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:19.377165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a4325-3ea7-42f3-853d-76da6c417bde None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:19.377966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a4325-3ea7-42f3-853d-76da6c417bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:19.408263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-354a4325-3ea7-42f3-853d-76da6c417bde None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:19.408263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-354a4325-3ea7-42f3-853d-76da6c417bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:19.594050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:19.596725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:19.596725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:19.600561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:19.610557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:19.648289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 902/1827] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:19.654105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:19.657382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:19.657611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:19.657867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:19.658615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:19.691240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 926/1828] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:19.695994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:19.699494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:19.717500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 903/1829] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:19.723349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:19.738910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73a6e5-537b-4d15-acb0-dc22f74443a2 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:19.747377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:19.748413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:19.748862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:19.749295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:19.750367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:19.750887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:19.756911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:19.781270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 927/1830] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:29:19.785471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:19.803303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad82b3-c245-445c-9391-c4b807422c4a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:19.813849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:19.814871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:19.815494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:19.816885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:19.818005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:19.820886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:19.928034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 904/1831] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:19 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:19.931641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:19.949859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5a376-b234-43de-9bec-ad00a3b4a777 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:20.096636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:20.178373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:20.179290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:20.179691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:20.184733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:20.186652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:20.189467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:20.251383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 928/1832] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:19 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:20.256987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:20.334664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292e924-472d-45e3-bbd2-7dd8a18a5976 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:20.355760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:20.363868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:20.364468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:20.364960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:20.366566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:20.369269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:20.406675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 905/1833] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:20.412731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:20.435920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae413932-33b9-4b96-be55-1fbba1ffbdd1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:20.443785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27438b4b-32e4-41d2-a5fc-b1ac095a7bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:20.445011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27438b4b-32e4-41d2-a5fc-b1ac095a7bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:20.445792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27438b4b-32e4-41d2-a5fc-b1ac095a7bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:20.446474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27438b4b-32e4-41d2-a5fc-b1ac095a7bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:20.448176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27438b4b-32e4-41d2-a5fc-b1ac095a7bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:20.453522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27438b4b-32e4-41d2-a5fc-b1ac095a7bfb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:20.476598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 929/1834] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:20.485544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:20.515514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924438-f238-43b7-9e4c-9c3cdc10db9a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:20.525147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:20.527261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:20.527876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:20.528571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] PATH_INFO: `/v3/projects/3a9cd9f3e7b949cf860c71d0523e8b3a/users/3e2700ea9be74da9abae344880213d51/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:20.562472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9cd9f3e7b949cf860c71d0523e8b3a, user_id=3e2700ea9be74da9abae344880213d51, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:20.568478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:20.599007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 906/1835] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:20 2022] PUT /identity/v3/projects/3a9cd9f3e7b949cf860c71d0523e8b3a/users/3e2700ea9be74da9abae344880213d51/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:20.634998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:20.636065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:20.636621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:20.717931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] MFA Rules not processed for user `3e2700ea9be74da9abae344880213d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:20.721007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:20.980349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 930/1836] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:20.993294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:21.194682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e93e-de28-406e-8c56-0364e31b51e5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:21.220485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:21.223793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:21.224524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:21.225014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:21.227187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:21.231449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:21.307718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 907/1837] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:20 2022] POST /identity/v3/projects => generated 383 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:21.313881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:21.343053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3e059-2279-4f44-833f-abc5b166b745 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:21.496299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:21.499571 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:21.502746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:21.503281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:21.509048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:21.517595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:21.587534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 931/1838] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:21 2022] POST /identity/v3/users => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:21.594288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:21.621074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42947a35-0a3f-4d4b-bcb1-ae392a5a27e1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:21.632580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:21.633751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:21.635294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:21.636168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:21.637198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:21.653738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:21.709265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:21.724546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 908/1839] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:21.749344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bae67-dfe3-4959-aced-d1f317053987 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:21.769117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:21.770717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:21.771167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:21.771598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] PATH_INFO: `/v3/projects/2ffe0ee66dab45b99bbdbf8e72a9f958/users/0cdbbc9729d44fb8a1f4cbc70f7dbffc/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:21.799840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffe0ee66dab45b99bbdbf8e72a9f958, user_id=0cdbbc9729d44fb8a1f4cbc70f7dbffc, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:21.803267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:21.834174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 932/1840] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:21 2022] PUT /identity/v3/projects/2ffe0ee66dab45b99bbdbf8e72a9f958/users/0cdbbc9729d44fb8a1f4cbc70f7dbffc/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:21.840482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:21.857312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:21.860320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:21.872602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:21.873854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a562692-9067-476b-8c8d-4e675e11059a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:21.891052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:21.954975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd2fde-d32d-4b7e-bc63-f2d1c77ce06b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:21.966899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:21.967923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:21.968442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:21.969172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:21.974455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:22.069624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:22.070347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:22.070639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:22.071001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:22.072281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:22.080250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:22.084877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:22.086786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5223b4-baec-41e6-a18b-414c7a56b7cb service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:22.096769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 909/1841] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:22.101142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:22.101581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 933/1842] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:22.123138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc124210-7d17-4c70-89c4-1ba9dc2eab09 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:22.132542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:22.134095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:22.134615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:22.135160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] PATH_INFO: `/v3/projects/2ffe0ee66dab45b99bbdbf8e72a9f958/users/0cdbbc9729d44fb8a1f4cbc70f7dbffc/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:22.140520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffe0ee66dab45b99bbdbf8e72a9f958, user_id=0cdbbc9729d44fb8a1f4cbc70f7dbffc, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:22.146230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:22.184618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 910/1843] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:22 2022] PUT /identity/v3/projects/2ffe0ee66dab45b99bbdbf8e72a9f958/users/0cdbbc9729d44fb8a1f4cbc70f7dbffc/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:22.189239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:22.190101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:22.190357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:22.257944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] MFA Rules not processed for user `0cdbbc9729d44fb8a1f4cbc70f7dbffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:22.263319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:22.469432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 934/1844] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:22.495606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:22.498303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:22.661365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2b12d-5e6d-44fe-9a93-4e68c2e29aeb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:22.670771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc477a7-5b10-4c91-ba18-d4f937c4c4da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:22.671582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc477a7-5b10-4c91-ba18-d4f937c4c4da service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:22.671960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc477a7-5b10-4c91-ba18-d4f937c4c4da service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:22.672346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc477a7-5b10-4c91-ba18-d4f937c4c4da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:22.790261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc477a7-5b10-4c91-ba18-d4f937c4c4da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:22.792126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc477a7-5b10-4c91-ba18-d4f937c4c4da service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:22.804109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 911/1845] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:29.639732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:29.642226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:29.747789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:29.780570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef2118-2b6d-4dde-a60c-536b104fb41d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:29.789222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:29.791404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:29.791404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:29.792345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] PATH_INFO: `/v3/users/36e17c6ceacc4ec0bc543b34fde28053` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:29.796110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e17c6ceacc4ec0bc543b34fde28053)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:29.799987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:29.879170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e17c6ceacc4ec0bc543b34fde28053'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:29.897156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e17c6ceacc4ec0bc543b34fde28053'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:29.914430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e17c6ceacc4ec0bc543b34fde28053'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:29.915529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e17c6ceacc4ec0bc543b34fde28053'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:29.916574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e17c6ceacc4ec0bc543b34fde28053'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:29.943419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 935/1846] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:29 2022] DELETE /identity/v3/users/36e17c6ceacc4ec0bc543b34fde28053 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:29.946869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:29.947540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:29.947540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:29.992627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:29.996543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:30.097805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:30.140697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 912/1847] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:30.160466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:30.161994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:30.377103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdb70-a554-42e7-bf5b-8bdb532b62fa admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:30.390949 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8683909-512f-4b73-9828-e9638747dedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:30.392023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683909-512f-4b73-9828-e9638747dedf service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:30.392519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683909-512f-4b73-9828-e9638747dedf service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:30.392858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683909-512f-4b73-9828-e9638747dedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:30.422837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8683909-512f-4b73-9828-e9638747dedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:30.428666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8683909-512f-4b73-9828-e9638747dedf service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:30.467242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 936/1848] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:31.107588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:31.113957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:31.214274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a9d2-9b73-4e1a-a73d-37ba6635b637 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:31.220753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:31.221837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:31.222129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:31.222407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:31.224950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:31.327617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:31.329536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d344d8-809a-437e-aa9c-fbc50dc9a736 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:31.354989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 913/1849] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:31.562362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8a1d7-e096-4c67-9442-e86c36f4c524 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:31.563011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8a1d7-e096-4c67-9442-e86c36f4c524 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:31.563680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8a1d7-e096-4c67-9442-e86c36f4c524 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:31.567601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 937/1850] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:29:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:29:31.633556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:31.633767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:31.633976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:31.677626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:31.680175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:32.015633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 914/1851] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:32.490383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8a1d7-e096-4c67-9442-e86c36f4c524 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:32.496240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-92d8a1d7-e096-4c67-9442-e86c36f4c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:32.549650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8a1d7-e096-4c67-9442-e86c36f4c524 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:32.568913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:32.570348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:32.573387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:32.574263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:32.597490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:32.602650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:32.632295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 938/1852] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:32.703562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:32.706993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:32.740068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382b0b5-c823-4705-ae24-33ca010748c9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:32.749175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:32.750205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:32.750648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:32.751249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] PATH_INFO: `/v3/projects/ae54f47737a64a129a8f63bdd666d945` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:32.752924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae54f47737a64a129a8f63bdd666d945)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:32.754994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:32.810711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:32.870152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae54f47737a64a129a8f63bdd666d945', 'request_id': 'req-b6c8259d-eb01-48a0-8e46-1c4840135360'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:32.881494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae54f47737a64a129a8f63bdd666d945', 'request_id': 'req-b6c8259d-eb01-48a0-8e46-1c4840135360'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:32.897781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae54f47737a64a129a8f63bdd666d945', 'request_id': 'req-b6c8259d-eb01-48a0-8e46-1c4840135360'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:32.950987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] The token cache is being invalidate because project ae54f47737a64a129a8f63bdd666d945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:32.951181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:32.952400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:32.959576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 915/1853] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:32 2022] DELETE /identity/v3/projects/ae54f47737a64a129a8f63bdd666d945 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:32.961596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:32.992588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec7a7b-98dc-4c94-a563-5a12171a866b service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:33.209507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:33.258662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:33.260911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:33.262060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:33.262805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] PATH_INFO: `/v3/users/0c9b9f0b79f54cdcacc6ea042845c3b1` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:33.266225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9b9f0b79f54cdcacc6ea042845c3b1)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:33.269710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:33.417898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9b9f0b79f54cdcacc6ea042845c3b1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:33.457702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9b9f0b79f54cdcacc6ea042845c3b1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:33.489037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9b9f0b79f54cdcacc6ea042845c3b1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:33.493213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9b9f0b79f54cdcacc6ea042845c3b1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:33.495421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9b9f0b79f54cdcacc6ea042845c3b1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:33.547437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 939/1854] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:32 2022] DELETE /identity/v3/users/0c9b9f0b79f54cdcacc6ea042845c3b1 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:34.605885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:34.608913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:34.634706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8259d-eb01-48a0-8e46-1c4840135360 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:34.749861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:34.752185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:34.752491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:34.752943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] PATH_INFO: `/v3/projects/18602dd042284868a6cd8fcfd1116815` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:34.754767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=18602dd042284868a6cd8fcfd1116815)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:34.757830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:34.879518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18602dd042284868a6cd8fcfd1116815', 'request_id': 'req-8e2bdd36-cf05-419c-8d89-d98962acee4e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:34.889456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18602dd042284868a6cd8fcfd1116815', 'request_id': 'req-8e2bdd36-cf05-419c-8d89-d98962acee4e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:34.898613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18602dd042284868a6cd8fcfd1116815', 'request_id': 'req-8e2bdd36-cf05-419c-8d89-d98962acee4e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:34.918580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] The token cache is being invalidate because project 18602dd042284868a6cd8fcfd1116815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:34.918880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:34.920105 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:34.922566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 916/1855] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:34 2022] DELETE /identity/v3/projects/18602dd042284868a6cd8fcfd1116815 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:34.926500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:34.928795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:34.951325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b6fe-fc1c-44a3-9fea-092b83b89b8e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:35.082119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:35.082119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:35.082119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:35.082583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] PATH_INFO: `/v3/users/7b5a77e267084f18918aa8c11d1495a4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:35.084011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5a77e267084f18918aa8c11d1495a4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:35.086230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:35.184582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5a77e267084f18918aa8c11d1495a4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:35.221941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5a77e267084f18918aa8c11d1495a4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:35.244472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5a77e267084f18918aa8c11d1495a4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:35.248754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5a77e267084f18918aa8c11d1495a4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:35.253042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-de80d90b-fb31-4fd7-a985-40ca004cb8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5a77e267084f18918aa8c11d1495a4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:35.256903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:35.293805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 940/1856] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:34 2022] DELETE /identity/v3/users/7b5a77e267084f18918aa8c11d1495a4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:36.379408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:36.382680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:36.412735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bdd36-cf05-419c-8d89-d98962acee4e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:36.559207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:36.606461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:36.608087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:36.608569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:36.609031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] PATH_INFO: `/v3/projects/5fa2f314b54749e9b2e397ffe5bd6683` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:36.612281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa2f314b54749e9b2e397ffe5bd6683)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:36.616205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:36.721016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa2f314b54749e9b2e397ffe5bd6683', 'request_id': 'req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:36.728133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa2f314b54749e9b2e397ffe5bd6683', 'request_id': 'req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:36.737050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa2f314b54749e9b2e397ffe5bd6683', 'request_id': 'req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:36.764323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] The token cache is being invalidate because project 5fa2f314b54749e9b2e397ffe5bd6683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:36.764869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:36.766267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b7add67a-70b6-4890-8b5b-5f7fd40cce3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:36.772401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 917/1857] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:36 2022] DELETE /identity/v3/projects/5fa2f314b54749e9b2e397ffe5bd6683 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:36.780006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:36.780874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:36.780874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:36.807948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:36.809814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:36.906423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 941/1858] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:36.920720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:36.921193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:36.921622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:36.945789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:36.947080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:36.958017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 918/1859] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:36.963964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:36.979499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609e40e-7442-4fb7-a0ad-457ad274ad39 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:36.991106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:36.992348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:36.992734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:36.995706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:36.995706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:36.995706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:36.997955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.023645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 942/1860] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:29:37.025410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.040597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88cef0-ab0d-4eec-bee8-42b951e72dfd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:37.048754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:37.050118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.050451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.050745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.051467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:37.053500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.108258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 919/1861] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:37 2022] POST /identity/v3/projects => generated 407 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:37.114036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.133327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab1c3-38f1-4448-8c7b-7f964016bd2e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:37.254466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:37.255371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.255669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.256023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.257081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:37.259243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.288878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 943/1862] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:37 2022] POST /identity/v3/users => generated 304 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:37.296257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.314934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6b05a-4580-498e-869c-f46f44d6d3a3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:37.321563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:37.322871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.323136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.323488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.324496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:37.326569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.341563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 920/1863] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:37.345906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.358210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeb204-287d-43c7-a186-3f8c8ce8d03a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:37.365481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fd184-7b39-46e7-b189-43a947a89579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:37.366096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd184-7b39-46e7-b189-43a947a89579 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.366384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd184-7b39-46e7-b189-43a947a89579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.366682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd184-7b39-46e7-b189-43a947a89579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.367416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fd184-7b39-46e7-b189-43a947a89579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:37.369104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fd184-7b39-46e7-b189-43a947a89579 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.382000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 944/1864] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:37.387602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.404007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480735-e9a5-4465-94d7-572b5706fa8a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:37.411758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:37.413515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.414621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.414621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] PATH_INFO: `/v3/projects/623778d0000d4b91a4205bdbe75595f4/users/fd088e4f593a4d0bb57dff32c953aa3a/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.450984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=623778d0000d4b91a4205bdbe75595f4, user_id=fd088e4f593a4d0bb57dff32c953aa3a, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:37.453661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.482487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 921/1865] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:37 2022] PUT /identity/v3/projects/623778d0000d4b91a4205bdbe75595f4/users/fd088e4f593a4d0bb57dff32c953aa3a/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:37.493776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.494058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.494516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.535026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] MFA Rules not processed for user `fd088e4f593a4d0bb57dff32c953aa3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:37.536346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:37.600964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:37.667389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 945/1866] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:37.670210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.809166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03312e-328c-4aec-9437-32658b6be285 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:37.817782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:37.819008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:37.819530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:37.820007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:37.821421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:37.826003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:37.855930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:37.883912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 922/1867] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:37 2022] POST /identity/v3/projects => generated 409 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:37.890874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:37.911059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d5ed1-7c77-4f57-a451-d76e45b258cc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:38.041225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:38.042546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.043363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.043691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.045164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:38.048016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:38.078437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 946/1868] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:37 2022] POST /identity/v3/users => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:38.083446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:38.106994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e715ca-b86e-4522-bca2-78edaeff7233 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:38.116009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:38.117393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.118022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.118782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.120447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:38.123991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:38.152886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 923/1869] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:38.154394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:38.173876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babb0b2-64dc-435c-9bc4-3596d52a33ec admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:38.183281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:38.185005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.185529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.186043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] PATH_INFO: `/v3/projects/ab9074e8d754409381411c80401b55ea/users/f2d14e7b7d0b483e8fd064a2c50802f7/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.213955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9074e8d754409381411c80401b55ea, user_id=f2d14e7b7d0b483e8fd064a2c50802f7, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:38.217588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:38.235938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 947/1870] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:38 2022] PUT /identity/v3/projects/ab9074e8d754409381411c80401b55ea/users/f2d14e7b7d0b483e8fd064a2c50802f7/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:38.242046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:38.255204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6658ae-8b67-461d-a89e-fe70b3a6fa8e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:38.371243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd81577-1621-43d1-8469-c14075af8eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:38.372537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd81577-1621-43d1-8469-c14075af8eec admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.373041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd81577-1621-43d1-8469-c14075af8eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.373532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd81577-1621-43d1-8469-c14075af8eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.374928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd81577-1621-43d1-8469-c14075af8eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:38.378201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd81577-1621-43d1-8469-c14075af8eec admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:38.403566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 924/1871] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:38.405426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:38.426817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd5ede-8816-4d84-aef8-44669abdda7d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:38.434320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:38.435258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.435542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.435878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] PATH_INFO: `/v3/projects/ab9074e8d754409381411c80401b55ea/users/f2d14e7b7d0b483e8fd064a2c50802f7/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.438241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9074e8d754409381411c80401b55ea, user_id=f2d14e7b7d0b483e8fd064a2c50802f7, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:38.441078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:38.457981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 948/1872] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:38 2022] PUT /identity/v3/projects/ab9074e8d754409381411c80401b55ea/users/f2d14e7b7d0b483e8fd064a2c50802f7/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:38.478533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.479643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.480200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.529638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] MFA Rules not processed for user `f2d14e7b7d0b483e8fd064a2c50802f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:38.531109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:38.645867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 925/1873] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:38.660665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:38.662661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:38.762971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d996-1453-4c1d-b6e5-826488b08e2c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:38.768082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:38.768967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:38.769375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:38.769661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:38.789669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:38.789911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:38.812630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 949/1874] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:39.187689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:39.190234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:39.240019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6483cc-423d-4ade-8d8d-ae9bd1e0c2f2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:39.247852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ab90-a1e5-4bb7-9f16-f86412583d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:39.249332 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ab90-a1e5-4bb7-9f16-f86412583d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:39.250520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ab90-a1e5-4bb7-9f16-f86412583d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:39.251073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ab90-a1e5-4bb7-9f16-f86412583d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:39.429626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ab90-a1e5-4bb7-9f16-f86412583d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:39.444932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ab90-a1e5-4bb7-9f16-f86412583d73 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:39.484346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 926/1875] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:39 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:39.887690 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:39.890776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:40.030751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:40.145848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804695f-8615-45d8-86f1-f0db9c5d336d service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:40.169237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:40.170945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:40.171396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:40.171873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] PATH_INFO: `/v3/users/3e2700ea9be74da9abae344880213d51` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:40.176230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2700ea9be74da9abae344880213d51)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:40.187543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:40.401834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2700ea9be74da9abae344880213d51'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:40.477582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2700ea9be74da9abae344880213d51'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:40.498489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2700ea9be74da9abae344880213d51'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:40.502090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2700ea9be74da9abae344880213d51'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:40.504864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2700ea9be74da9abae344880213d51'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:40.555416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 950/1876] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:39 2022] DELETE /identity/v3/users/3e2700ea9be74da9abae344880213d51 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:40.558629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:40.559222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:40.559697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:40.592919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:40.649404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:40.658568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:40.901753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 927/1877] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:40.915452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:40.917856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:41.053658 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e84b-4057-40a0-95ae-b21746f90e51 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:41.061001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:41.062297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:41.062927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:41.063501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:41.094009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:41.098679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:41.119118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 951/1878] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:41.956597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:41.958263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:41.984394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6a94-b811-4361-b5e8-6c646b4f80cb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:42.005657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:42.007000 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:42.007429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:42.007850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] PATH_INFO: `/v3/projects/3a9cd9f3e7b949cf860c71d0523e8b3a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:42.011046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9cd9f3e7b949cf860c71d0523e8b3a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:42.021238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:42.227603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9cd9f3e7b949cf860c71d0523e8b3a', 'request_id': 'req-21c76b98-e327-461d-be47-3986e33de715'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.246974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9cd9f3e7b949cf860c71d0523e8b3a', 'request_id': 'req-21c76b98-e327-461d-be47-3986e33de715'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.264439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9cd9f3e7b949cf860c71d0523e8b3a', 'request_id': 'req-21c76b98-e327-461d-be47-3986e33de715'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.305882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] The token cache is being invalidate because project 3a9cd9f3e7b949cf860c71d0523e8b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:42.306483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.308899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.311549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 928/1879] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:41 2022] DELETE /identity/v3/projects/3a9cd9f3e7b949cf860c71d0523e8b3a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:42.317935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:42.343751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a3050-370e-4f31-a1e5-a3903777db02 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:42.460883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:42.461887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:42.462242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:42.462584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] PATH_INFO: `/v3/users/0cdbbc9729d44fb8a1f4cbc70f7dbffc` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:42.465227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cdbbc9729d44fb8a1f4cbc70f7dbffc)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:42.467610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:42.473901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:42.568889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cdbbc9729d44fb8a1f4cbc70f7dbffc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.598758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cdbbc9729d44fb8a1f4cbc70f7dbffc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.612460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cdbbc9729d44fb8a1f4cbc70f7dbffc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.614023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cdbbc9729d44fb8a1f4cbc70f7dbffc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.615201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cdbbc9729d44fb8a1f4cbc70f7dbffc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:42.643664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 952/1880] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:42 2022] DELETE /identity/v3/users/0cdbbc9729d44fb8a1f4cbc70f7dbffc => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:42.847077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:42.849275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:42.881664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76b98-e327-461d-be47-3986e33de715 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:43.118054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:43.121054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:43.234178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:43.237501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.238337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.239084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] PATH_INFO: `/v3/users/a3072df3bc394dd6ab3dd14cd24c1d50` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.242063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3072df3bc394dd6ab3dd14cd24c1d50)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:43.247815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:43.323851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:43.346916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05d3a-0955-476a-9dbb-2feed3c5b478 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:43.356897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:43.358585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.359128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.359634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] PATH_INFO: `/v3/projects/2ffe0ee66dab45b99bbdbf8e72a9f958` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.362188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffe0ee66dab45b99bbdbf8e72a9f958)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:43.364579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3072df3bc394dd6ab3dd14cd24c1d50'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.366168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:43.377966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3072df3bc394dd6ab3dd14cd24c1d50'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.393577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3072df3bc394dd6ab3dd14cd24c1d50'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.424206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3072df3bc394dd6ab3dd14cd24c1d50'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.425626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5dfc284d-c3a2-4bed-8649-5ca1e54d2246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3072df3bc394dd6ab3dd14cd24c1d50'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.439090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 929/1881] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:42 2022] DELETE /identity/v3/users/a3072df3bc394dd6ab3dd14cd24c1d50 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:43.445647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.446076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.446686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.477899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:43.480717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:43.500741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffe0ee66dab45b99bbdbf8e72a9f958', 'request_id': 'req-29b7fef8-0c83-4be2-adea-85e725d88720'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.515425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffe0ee66dab45b99bbdbf8e72a9f958', 'request_id': 'req-29b7fef8-0c83-4be2-adea-85e725d88720'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.532907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffe0ee66dab45b99bbdbf8e72a9f958', 'request_id': 'req-29b7fef8-0c83-4be2-adea-85e725d88720'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.563504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] The token cache is being invalidate because project 2ffe0ee66dab45b99bbdbf8e72a9f958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:43.563985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.565168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b7fef8-0c83-4be2-adea-85e725d88720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:43.568363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 953/1882] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:43 2022] DELETE /identity/v3/projects/2ffe0ee66dab45b99bbdbf8e72a9f958 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:43.578023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fd66a-6d73-49a3-b983-1df72f182176 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.578498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fd66a-6d73-49a3-b983-1df72f182176 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.578999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fd66a-6d73-49a3-b983-1df72f182176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.617896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-711fd66a-6d73-49a3-b983-1df72f182176 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:43.621182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-711fd66a-6d73-49a3-b983-1df72f182176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:43.638704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:43.644935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 930/1883] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:43.657590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:43.659595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:43.742639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a258-59e0-48ce-9d98-3afc69d12f16 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:43.745586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 954/1884] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:43.751943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:43.753084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.753463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.753641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.761984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db077f-73e5-447d-8f61-2441333de102 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.762457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db077f-73e5-447d-8f61-2441333de102 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.762912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db077f-73e5-447d-8f61-2441333de102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.766673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:43.768144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:43.782711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 931/1885] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:43.800106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-46db077f-73e5-447d-8f61-2441333de102 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:43.802325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-46db077f-73e5-447d-8f61-2441333de102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:43.829327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 955/1886] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:43.836318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:43.865739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54deed9-a017-4c6b-b572-db7faa22a406 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:43.876210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:43.878124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.878616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.879110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.880687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:43.881372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:43.885832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:43.913947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 932/1887] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:29:43.920947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db077f-73e5-447d-8f61-2441333de102 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:43.937410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db077f-73e5-447d-8f61-2441333de102 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:43.950779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:43.951899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:43.953599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:43.954333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:43.956727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:43.960817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.026153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 956/1888] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:43 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:44.036566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.056631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c58dd9-5bec-4599-bd1d-1f2979dfe3c3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.115049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:44.157726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.158526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.158806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.159510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.160025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.162279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.192532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 933/1889] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:44 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:44.194089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.209151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e255-2754-456c-8903-5ad12aa1012b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.216491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.218196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.218487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.218763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.219960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.222002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.240001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 957/1890] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:44.242162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.272755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8ad1-2d1e-480f-8645-614d74ad6e81 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.281306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.282896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.283583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.283997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.285496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.288953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.318900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 934/1891] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:44.325740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.334466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.337796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:44.344284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcbad9-473a-4fb4-994f-171c56ebdb24 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.350708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b232a26-4ab0-43d6-a943-88958e89db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.362327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30b9d-bc94-458e-b18b-960e1478e2e3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.365330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b232a26-4ab0-43d6-a943-88958e89db98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.365807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b232a26-4ab0-43d6-a943-88958e89db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.366290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b232a26-4ab0-43d6-a943-88958e89db98 admin admin] PATH_INFO: `/v3/projects/40897ba73b574fcb8c13a1410e6b8d44/users/d3fba333ea45449c854cbe632bc853d9/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.370868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.371231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.371556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.371780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] PATH_INFO: `/v3/projects/0d0cf5a8cccb491395ffceb4018300f9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.373166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0cf5a8cccb491395ffceb4018300f9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.375636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.398333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b232a26-4ab0-43d6-a943-88958e89db98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40897ba73b574fcb8c13a1410e6b8d44, user_id=d3fba333ea45449c854cbe632bc853d9, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.401509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b232a26-4ab0-43d6-a943-88958e89db98 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.419259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 958/1892] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:44 2022] PUT /identity/v3/projects/40897ba73b574fcb8c13a1410e6b8d44/users/d3fba333ea45449c854cbe632bc853d9/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:44.434267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.434733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.435190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.462256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0cf5a8cccb491395ffceb4018300f9', 'request_id': 'req-ed285f3c-ceb8-4c02-bed6-352d3457215e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.478891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] MFA Rules not processed for user `d3fba333ea45449c854cbe632bc853d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:44.480714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0cf5a8cccb491395ffceb4018300f9', 'request_id': 'req-ed285f3c-ceb8-4c02-bed6-352d3457215e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.480958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:44.495991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0cf5a8cccb491395ffceb4018300f9', 'request_id': 'req-ed285f3c-ceb8-4c02-bed6-352d3457215e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.529455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] The token cache is being invalidate because project 0d0cf5a8cccb491395ffceb4018300f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:44.529911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.531085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.535013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 935/1893] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:44 2022] DELETE /identity/v3/projects/0d0cf5a8cccb491395ffceb4018300f9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:44.544164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.546658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:44.624317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 959/1894] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:44.626777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.628435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:44.671295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed285f3c-ceb8-4c02-bed6-352d3457215e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.678348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.679585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.680200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.680606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] PATH_INFO: `/v3/users/659c93b25f374a54800eec0f2b1a43d9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.683008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659c93b25f374a54800eec0f2b1a43d9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.685400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.698992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:44.731527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558982a8-8590-4374-a7bf-036c9f95d3ed None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:44.738153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:44.739011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:44.739414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:44.739780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:44.740452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:44.741050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:44.744505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:44.752604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659c93b25f374a54800eec0f2b1a43d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.775313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659c93b25f374a54800eec0f2b1a43d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.793283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659c93b25f374a54800eec0f2b1a43d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.794494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 960/1895] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:44 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:44.799047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:44.820661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659c93b25f374a54800eec0f2b1a43d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.830656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659c93b25f374a54800eec0f2b1a43d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:44.859877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 936/1896] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:44 2022] DELETE /identity/v3/users/659c93b25f374a54800eec0f2b1a43d9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:44.923611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611f715-bd6f-4f45-b106-4af1805fecf3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.013225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.014193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.014747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.015188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.016419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.018691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.051941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 961/1897] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:44 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:45.056479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.059303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:45.081387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92a15c-6ef2-46b5-a591-6e05a338cc09 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.087750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.089201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.089661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.090163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.091171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.093193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.107940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 937/1898] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:45.112644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.125291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390c65-8751-4b96-b2db-472b8ce4788d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.131176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbcdbc-7a14-4d42-b9bf-9cd311c5b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.131973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbcdbc-7a14-4d42-b9bf-9cd311c5b0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.132291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbcdbc-7a14-4d42-b9bf-9cd311c5b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.132784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbcdbc-7a14-4d42-b9bf-9cd311c5b0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.133700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbcdbc-7a14-4d42-b9bf-9cd311c5b0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.136279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbcdbc-7a14-4d42-b9bf-9cd311c5b0d6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.152364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 962/1899] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:45.156733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.170129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabdd26-f118-4f1c-b4a6-a87bba13c17b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.175934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.177096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.177479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.177479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] PATH_INFO: `/v3/projects/80ad90bc11be4eb9b52a65e548422aca/users/4f927d9891194d7b9b0f3eb769b5dd73/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.201527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ad90bc11be4eb9b52a65e548422aca, user_id=4f927d9891194d7b9b0f3eb769b5dd73, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.204610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.226548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 938/1900] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:45 2022] PUT /identity/v3/projects/80ad90bc11be4eb9b52a65e548422aca/users/4f927d9891194d7b9b0f3eb769b5dd73/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:45.238789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.239238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.239523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.279469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] MFA Rules not processed for user `4f927d9891194d7b9b0f3eb769b5dd73`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:45.281764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:45.397532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.401448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:45.401964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:45.466145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 963/1901] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:45.468117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.538317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cc889-3b03-4283-ad22-f7d1d46093a2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.547991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.549501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.550345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.550844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] PATH_INFO: `/v3/projects/7f3294df07ac4246bdfe72b90677cdf8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.553514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3294df07ac4246bdfe72b90677cdf8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.561915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.577499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b5250-6dec-4e18-9fcf-df5e04ab0488 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.583916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.584920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.585480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.585911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.586792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.589871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.632659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 964/1902] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:45 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:45.633834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.638472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3294df07ac4246bdfe72b90677cdf8', 'request_id': 'req-2418fdd7-7b70-4a17-8487-7069f7328d8f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:45.645611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3294df07ac4246bdfe72b90677cdf8', 'request_id': 'req-2418fdd7-7b70-4a17-8487-7069f7328d8f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:45.654323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3294df07ac4246bdfe72b90677cdf8', 'request_id': 'req-2418fdd7-7b70-4a17-8487-7069f7328d8f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:45.671174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] The token cache is being invalidate because project 7f3294df07ac4246bdfe72b90677cdf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:45.671993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:45.672268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2418fdd7-7b70-4a17-8487-7069f7328d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:45.674216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 939/1903] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:45 2022] DELETE /identity/v3/projects/7f3294df07ac4246bdfe72b90677cdf8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:45.680926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d69b5-5026-4cfc-9e68-a327b7f02430 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.681271 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d69b5-5026-4cfc-9e68-a327b7f02430 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.681602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d69b5-5026-4cfc-9e68-a327b7f02430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.713829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c45d69b5-5026-4cfc-9e68-a327b7f02430 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:45.715985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c45d69b5-5026-4cfc-9e68-a327b7f02430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:45.728610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:45.748484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56689aad-3b64-4c79-ab51-762958b6c569 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.761490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.762669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.763129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.763540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.764962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.768580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.810736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 965/1904] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:45 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:45.814799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.818060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:45.835205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf84be1-3df9-436a-a0a9-3faf7415e67c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.849487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.850614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.851035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.851456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.852781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.863914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.892625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 940/1905] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:45.894220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 966/1906] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:45.899855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.901906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.902935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.903225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.920646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927da46f-d5aa-4a7d-a94d-d7b514231e13 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:45.937651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:45.938595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:45.938879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:45.939167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] PATH_INFO: `/v3/projects/31a99c42b19a46e2805cb68137ea50ec/users/14a5711bb1ab4e6d80b207732a98c764/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:45.941090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:45.943147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:45.970492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 941/1907] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:45.973044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a99c42b19a46e2805cb68137ea50ec, user_id=14a5711bb1ab4e6d80b207732a98c764, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:45.976267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:45.982206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:45.996151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4a351-3179-4682-ab6b-284bcfd36ff8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.014756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 967/1908] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:45 2022] PUT /identity/v3/projects/31a99c42b19a46e2805cb68137ea50ec/users/14a5711bb1ab4e6d80b207732a98c764/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:46.015046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.016033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.016349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.016628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.017651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:46.017985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.019815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.022181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.074526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 942/1909] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:29:46.080975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.091932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:46.118614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10af0-e3e5-4390-99d1-e8925b64b14f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.241474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150178e-1aba-4185-a283-6c8c0c9bde53 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.253387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.254256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.254548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.254829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.255586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.257566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.272835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 968/1910] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:46.279743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.297111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.297832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.298182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.298458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.299166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.304954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.314354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a888c5-2c91-4028-a68b-99dccde26c0a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.339438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2112252-a57e-44f1-a2b9-e2c87ba931bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.344290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2112252-a57e-44f1-a2b9-e2c87ba931bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.345252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2112252-a57e-44f1-a2b9-e2c87ba931bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.347595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2112252-a57e-44f1-a2b9-e2c87ba931bb admin admin] PATH_INFO: `/v3/projects/31a99c42b19a46e2805cb68137ea50ec/users/14a5711bb1ab4e6d80b207732a98c764/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.360825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2112252-a57e-44f1-a2b9-e2c87ba931bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a99c42b19a46e2805cb68137ea50ec, user_id=14a5711bb1ab4e6d80b207732a98c764, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.369979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 943/1911] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:46 2022] POST /identity/v3/projects => generated 371 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:46.372394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2112252-a57e-44f1-a2b9-e2c87ba931bb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.374537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.399533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 969/1912] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:46 2022] PUT /identity/v3/projects/31a99c42b19a46e2805cb68137ea50ec/users/14a5711bb1ab4e6d80b207732a98c764/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:46.428232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.429605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.430036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.492379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6135-a877-45d1-bb35-7558ec044db2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.509792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] MFA Rules not processed for user `14a5711bb1ab4e6d80b207732a98c764`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:46.517579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:46.603342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:46.665818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 970/1913] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:46.680041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.681809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:46.715789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.718034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.718306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.718588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.722240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.724215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.772452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 944/1914] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:46 2022] POST /identity/v3/users => generated 286 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:46.774352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.793748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa8c5e-97bb-425b-9aa6-c9d1f27b51f6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.810325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.812176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.812881 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.813675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.815955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.819172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.826057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecde85-b9c9-4af8-99b6-5629321c2553 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.834881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.836124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.836739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.837260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.841054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 945/1915] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:46.844206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:46.845553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.859273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e882a0-1ada-41f1-86f1-2dff563533de admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.866622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.867645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.868605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.869061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.871672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.875148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.895276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 946/1916] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:46.896291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:46.918003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c82a71-d257-42f9-b8a7-3c7c88504e82 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:46.925841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3609-dd40-48bc-a288-c6e5f44dce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:46.926946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3609-dd40-48bc-a288-c6e5f44dce4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:46.927635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3609-dd40-48bc-a288-c6e5f44dce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:46.928120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3609-dd40-48bc-a288-c6e5f44dce4c admin admin] PATH_INFO: `/v3/projects/59f989a07645494eac5d7376951ff4ec/users/75b9f421163e4c888b9efaa74f220fac/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:46.955386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3609-dd40-48bc-a288-c6e5f44dce4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f989a07645494eac5d7376951ff4ec, user_id=75b9f421163e4c888b9efaa74f220fac, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:46.958587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3609-dd40-48bc-a288-c6e5f44dce4c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:46.963331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:47.003713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 947/1917] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:46 2022] PUT /identity/v3/projects/59f989a07645494eac5d7376951ff4ec/users/75b9f421163e4c888b9efaa74f220fac/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:47.004065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.006403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.007297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.009216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.010819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.025179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 971/1918] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:47.077137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] MFA Rules not processed for user `75b9f421163e4c888b9efaa74f220fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:47.080488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:47.241930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 948/1919] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:47.249621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:47.272179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22630e4b-e9cd-4d82-ab61-bc348129caa5 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:47.376525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:47.377911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.378193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.378277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.379142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.387295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.439018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 972/1920] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:47 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:47.446225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:47.461796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6e97e-2a4a-425a-a9f1-456d0eb7a7b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:47.566983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:47.578849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:47.579578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.579863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.580205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.581333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.583591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.610925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 949/1921] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:47 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:47.615340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:47.632264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf599b-a8f3-4a43-b035-3de5a0f96e8f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:47.639566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:47.640750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.641205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.641890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.643187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.646171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.669898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 973/1922] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:47.671663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:47.689450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aa25c-d856-4d7f-a243-4bb9640b7dfe admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:47.696416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:47.699840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.700379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.700887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] PATH_INFO: `/v3/projects/d67a19641da14564bbfaddd0c0e8f4be/users/ed237897638f4852a28b263885fee824/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.721562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67a19641da14564bbfaddd0c0e8f4be, user_id=ed237897638f4852a28b263885fee824, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.724352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.746009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 950/1923] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:47 2022] PUT /identity/v3/projects/d67a19641da14564bbfaddd0c0e8f4be/users/ed237897638f4852a28b263885fee824/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:47.752182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:47.768161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84fa14-63e7-463c-b8f9-965d187b0a1b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:47.836731 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cf6d3-62fd-49a0-9183-34d11912b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:47.838250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cf6d3-62fd-49a0-9183-34d11912b07b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.838654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cf6d3-62fd-49a0-9183-34d11912b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.838944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cf6d3-62fd-49a0-9183-34d11912b07b admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.839707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cf6d3-62fd-49a0-9183-34d11912b07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.841414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cf6d3-62fd-49a0-9183-34d11912b07b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.855231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 974/1924] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:47.859558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:47.872322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756e98-ec99-4d04-aefc-b1281a42a5e4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:47.877334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:47.878266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.878548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.878827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] PATH_INFO: `/v3/projects/d67a19641da14564bbfaddd0c0e8f4be/users/ed237897638f4852a28b263885fee824/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.881258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67a19641da14564bbfaddd0c0e8f4be, user_id=ed237897638f4852a28b263885fee824, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:47.884087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:47.898158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 951/1925] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:47 2022] PUT /identity/v3/projects/d67a19641da14564bbfaddd0c0e8f4be/users/ed237897638f4852a28b263885fee824/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:47.910359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:47.910653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:47.910939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:47.968615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] MFA Rules not processed for user `ed237897638f4852a28b263885fee824`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:47.970040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:48.067786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:48.125974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 975/1926] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:48.141226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:48.143557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:48.217428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aec6d2-91a7-4e1a-96ba-171010088837 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:48.222163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:48.222984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:48.223320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:48.223652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:48.235612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:48.237539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:48.250043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 952/1927] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:49.085408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:49.101824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f62b-90d2-4570-9abb-3595c20ded7f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:49.111327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:49.111573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:49.112928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:49.113408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:49.113836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:49.116346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:49.230695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809b5c3-cffd-4092-8b7d-43bc437731ac service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:49.236363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:49.237303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:49.237586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:49.238109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] PATH_INFO: `/v3/users/75b9f421163e4c888b9efaa74f220fac` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:49.238288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:49.239506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b9f421163e4c888b9efaa74f220fac)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:49.241129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a176b-a215-4500-9eae-6c8493993b42 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:49.241654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:49.271775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 976/1928] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:49.339006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b9f421163e4c888b9efaa74f220fac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.349538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:49.356210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b9f421163e4c888b9efaa74f220fac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.373557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b9f421163e4c888b9efaa74f220fac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.402050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b9f421163e4c888b9efaa74f220fac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.403143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b9f421163e4c888b9efaa74f220fac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.414753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 953/1929] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:49 2022] DELETE /identity/v3/users/75b9f421163e4c888b9efaa74f220fac => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:49.422166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:49.422685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:49.423002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:49.452042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:49.454206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:49.575102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:49.577411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:49.601479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 977/1930] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:49.609123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852d6c-475c-4f05-a438-3cae07ef7596 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:49.617016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:49.620669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:49.622584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:49.623911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:49.624508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:49.625155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] PATH_INFO: `/v3/users/fd088e4f593a4d0bb57dff32c953aa3a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:49.629212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd088e4f593a4d0bb57dff32c953aa3a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:49.633670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:49.731995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b55c2e-1ce7-4614-b965-9d4918606ef3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:49.743774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31c6f5-584f-4705-be20-b526b62ee91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:49.747075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31c6f5-584f-4705-be20-b526b62ee91d service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:49.748796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31c6f5-584f-4705-be20-b526b62ee91d service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:49.749513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31c6f5-584f-4705-be20-b526b62ee91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:49.755006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd088e4f593a4d0bb57dff32c953aa3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.774691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd088e4f593a4d0bb57dff32c953aa3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.789545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31c6f5-584f-4705-be20-b526b62ee91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:49.794236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31c6f5-584f-4705-be20-b526b62ee91d service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:49.800166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd088e4f593a4d0bb57dff32c953aa3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.825119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 978/1931] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:49.830579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd088e4f593a4d0bb57dff32c953aa3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.832200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd088e4f593a4d0bb57dff32c953aa3a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:49.848360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 954/1932] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:49 2022] DELETE /identity/v3/users/fd088e4f593a4d0bb57dff32c953aa3a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:49.851543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:49.855076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:49.855076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:49.921678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:49.923920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:49.960350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:50.082168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 979/1933] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:50.095875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:50.098088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:50.236578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc670-4581-41bd-9209-b7cdcd45aa56 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:50.241125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:50.241903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:50.242313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:50.243020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:50.257576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:50.261073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:50.276733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 955/1934] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:50.518392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:50.520966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:50.642738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6683-b1ed-4af7-bdeb-7fa3c1e99910 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:50.647683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:50.648474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:50.649051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:50.649526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:50.653194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:50.782493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:50.785008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:50.801078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 980/1935] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:50.813828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:50.815027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:50.851635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93418-ce66-4b77-9cfd-cc9c00e0dfe5 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:50.898234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:50.899796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:50.902696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:50.903281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] PATH_INFO: `/v3/projects/59f989a07645494eac5d7376951ff4ec` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:50.907630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f989a07645494eac5d7376951ff4ec)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:50.925229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:50.927109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:50.930419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:50.965223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152203b1-ea55-46ce-b546-32341214d88d service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:50.984838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:50.986244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:50.990207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:50.991135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] PATH_INFO: `/v3/projects/623778d0000d4b91a4205bdbe75595f4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:50.995204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=623778d0000d4b91a4205bdbe75595f4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:51.000476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:51.067137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=82,keystone:INSERT=2,keystone:DELETE=16 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:51.078317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f989a07645494eac5d7376951ff4ec', 'request_id': 'req-d2774ef6-7f88-4029-9187-eb5489e2a72c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.089888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f989a07645494eac5d7376951ff4ec', 'request_id': 'req-d2774ef6-7f88-4029-9187-eb5489e2a72c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.101522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f989a07645494eac5d7376951ff4ec', 'request_id': 'req-d2774ef6-7f88-4029-9187-eb5489e2a72c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.116198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '623778d0000d4b91a4205bdbe75595f4', 'request_id': 'req-8b48f850-052a-44cc-aa90-cdbda6c00a7c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.126986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] The token cache is being invalidate because project 59f989a07645494eac5d7376951ff4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:51.127657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.128693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.130725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 956/1936] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:50 2022] DELETE /identity/v3/projects/59f989a07645494eac5d7376951ff4ec => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:51.142143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:51.146216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:51.146752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '623778d0000d4b91a4205bdbe75595f4', 'request_id': 'req-8b48f850-052a-44cc-aa90-cdbda6c00a7c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.168725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '623778d0000d4b91a4205bdbe75595f4', 'request_id': 'req-8b48f850-052a-44cc-aa90-cdbda6c00a7c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.175566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:51.201697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] The token cache is being invalidate because project 623778d0000d4b91a4205bdbe75595f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:51.203127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.205808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.209371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 981/1937] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:50 2022] DELETE /identity/v3/projects/623778d0000d4b91a4205bdbe75595f4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:51.217947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:51.219758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:51.302572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774ef6-7f88-4029-9187-eb5489e2a72c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:51.379835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f850-052a-44cc-aa90-cdbda6c00a7c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:51.386131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:51.387664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:51.387956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:51.388246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] PATH_INFO: `/v3/users/f2d14e7b7d0b483e8fd064a2c50802f7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:51.389904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d14e7b7d0b483e8fd064a2c50802f7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:51.391887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:51.409828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:51.410981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:51.411496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:51.411866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] PATH_INFO: `/v3/users/ed237897638f4852a28b263885fee824` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:51.413474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed237897638f4852a28b263885fee824)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:51.415843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:51.478537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d14e7b7d0b483e8fd064a2c50802f7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.494704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed237897638f4852a28b263885fee824'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.524264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d14e7b7d0b483e8fd064a2c50802f7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.525180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed237897638f4852a28b263885fee824'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.538671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d14e7b7d0b483e8fd064a2c50802f7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.539846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed237897638f4852a28b263885fee824'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.540310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d14e7b7d0b483e8fd064a2c50802f7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.543894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d14e7b7d0b483e8fd064a2c50802f7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.571706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:51.574466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed237897638f4852a28b263885fee824'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.575518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a0173df6-26a8-48bb-9fb6-16e889674e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed237897638f4852a28b263885fee824'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:51.583359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 982/1938] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:51 2022] DELETE /identity/v3/users/f2d14e7b7d0b483e8fd064a2c50802f7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:51.589218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 957/1939] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:51 2022] DELETE /identity/v3/users/ed237897638f4852a28b263885fee824 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:52.357431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:52.359286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:52.534836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755433-5f90-4c75-9a16-65150ba20e97 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:52.575838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:52.700286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:52.701843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:52.702358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:52.702856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] PATH_INFO: `/v3/projects/d67a19641da14564bbfaddd0c0e8f4be` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:52.705292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d67a19641da14564bbfaddd0c0e8f4be)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:52.709212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:52.805070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd67a19641da14564bbfaddd0c0e8f4be', 'request_id': 'req-22c60a4d-4b9c-4238-93e7-00c20e678f10'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:52.817040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd67a19641da14564bbfaddd0c0e8f4be', 'request_id': 'req-22c60a4d-4b9c-4238-93e7-00c20e678f10'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:52.824924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd67a19641da14564bbfaddd0c0e8f4be', 'request_id': 'req-22c60a4d-4b9c-4238-93e7-00c20e678f10'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:52.841896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] The token cache is being invalidate because project d67a19641da14564bbfaddd0c0e8f4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:52.842224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:52.842915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-22c60a4d-4b9c-4238-93e7-00c20e678f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:52.844625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 983/1940] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:52 2022] DELETE /identity/v3/projects/d67a19641da14564bbfaddd0c0e8f4be => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:52.852975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:52.853846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:52.854564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:52.884257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:52.887270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:53.003679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 958/1941] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:53.019956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.020815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.021229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.057146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:53.059853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:53.061063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.063894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:53.076020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 984/1942] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:53.082856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.097673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563ffd7-a915-4749-9df2-531779aebe34 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.099599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd521b05-2e04-4b41-b4b1-b8250d6f589a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.106770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.108234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.108745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.109286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] PATH_INFO: `/v3/projects/ab9074e8d754409381411c80401b55ea` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.111264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.111792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9074e8d754409381411c80401b55ea)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.112540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.113047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.113597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.115001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:53.115581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.115867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.118759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.151494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 985/1943] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:29:53.158991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.198944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1aeb3-ffda-473f-bc44-154e901e6d1b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.209305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.210425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.210868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.211346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.213073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.217405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.300935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9074e8d754409381411c80401b55ea', 'request_id': 'req-4a0334b0-4a5b-410e-aa90-46edea1c18ba'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:53.318987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 986/1944] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:29:53.327030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.337068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9074e8d754409381411c80401b55ea', 'request_id': 'req-4a0334b0-4a5b-410e-aa90-46edea1c18ba'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:53.362672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9074e8d754409381411c80401b55ea', 'request_id': 'req-4a0334b0-4a5b-410e-aa90-46edea1c18ba'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:53.413942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] The token cache is being invalidate because project ab9074e8d754409381411c80401b55ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:53.414547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:53.415759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a0334b0-4a5b-410e-aa90-46edea1c18ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:53.419978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 959/1945] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:53 2022] DELETE /identity/v3/projects/ab9074e8d754409381411c80401b55ea => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:53.428436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6c2b7-13de-4294-81d9-ff66a844911d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.428952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6c2b7-13de-4294-81d9-ff66a844911d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.429425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6c2b7-13de-4294-81d9-ff66a844911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.471461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bda6c2b7-13de-4294-81d9-ff66a844911d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:53.473603 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecf07e-ffb2-4979-a597-14104ffaf409 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.473750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bda6c2b7-13de-4294-81d9-ff66a844911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:53.487485 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.489159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.489513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.489904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.490914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.494722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.496004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 960/1946] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:53.506162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.506497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.506818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.546194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:53.548016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:53.550439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:53.561579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 987/1947] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/users => generated 291 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:53.568013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.571626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:53.574902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 961/1948] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:53.580921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.595809 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417261f5-7608-4587-b816-595ebcbfe5b3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.603703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba836-816a-4004-99af-70beac89fd2f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.608298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.609831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.610584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.612209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.613718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:53.614245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.615687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.616818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.617291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.617712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.617841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.618998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.622820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.643798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 988/1949] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:53.658103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.659149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 962/1950] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:29:53.677534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.686206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd5d6-982e-46b2-8936-1e5e6bfa74e8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.698402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eced91e-51b5-4703-afda-1172422a47dd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.699310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.700096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.700359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.700619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.703930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.705685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.706838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.707731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.709101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.720903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.722393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.726078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 989/1951] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:53.728372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.733071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.757859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f85e2e-043f-4eb0-b072-28e5a0a422ed admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:53.777426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76eb32-08f4-4004-8d39-44149ef7503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:53.779296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76eb32-08f4-4004-8d39-44149ef7503d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.779798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76eb32-08f4-4004-8d39-44149ef7503d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.780348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76eb32-08f4-4004-8d39-44149ef7503d admin admin] PATH_INFO: `/v3/projects/bfdd3a05b6154d98a34c6c5c4453be2d/users/ea19e35727714e7f98170a6a34a64a54/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.801447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 963/1952] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:53.808347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:53.818647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76eb32-08f4-4004-8d39-44149ef7503d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfdd3a05b6154d98a34c6c5c4453be2d, user_id=ea19e35727714e7f98170a6a34a64a54, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:53.826522 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76eb32-08f4-4004-8d39-44149ef7503d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:53.872203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 990/1953] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:53 2022] PUT /identity/v3/projects/bfdd3a05b6154d98a34c6c5c4453be2d/users/ea19e35727714e7f98170a6a34a64a54/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:53.882833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:53.883714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:53.884429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:53.899701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:53.981723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] MFA Rules not processed for user `ea19e35727714e7f98170a6a34a64a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:53.988438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:54.001718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07c6a1-1980-45b0-812f-c14402b3e3c8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.205347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 991/1954] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:54.211055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:54.254287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:54.257224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.258066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.258703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.261355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:54.266684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:54.334172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 964/1955] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:53 2022] POST /identity/v3/users => generated 292 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:54.341532 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:54.374706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666956a0-3020-499d-b9e5-7760668c0475 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.377155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c2306-5d09-4aa1-aed6-02f45da80ceb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.382602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:54.383292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.383573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.383848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.384768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:54.388223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:54.392255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:54.394540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.397402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.397402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.404528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:54.406334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:54.441952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 965/1956] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:54.443336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:54.458760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d795b-211f-46b3-b09f-bb2805d383e0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.471059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 992/1957] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:54 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:54.474258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:54.486689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:54.488941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.489472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.490194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.491661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:54.499338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:54.548907 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 966/1958] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:54.566611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:54.613776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e153-bc31-4557-a077-975a18acd016 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.643616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:54.785431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef4568-a849-48ec-8f21-530aa0ab6cfd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.793933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f58bc-db38-43ba-b604-6fe944b30b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:54.795598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:54.796928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.797520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.797683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f58bc-db38-43ba-b604-6fe944b30b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.797849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.801228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f58bc-db38-43ba-b604-6fe944b30b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.801533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:54.801686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f58bc-db38-43ba-b604-6fe944b30b31 admin admin] PATH_INFO: `/v3/projects/fc5102d8b13d4212a2dbef0b5b199a5d/users/a390f532f2eb4da8af83a01d4397b6d8/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.801829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:54.845149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f58bc-db38-43ba-b604-6fe944b30b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5102d8b13d4212a2dbef0b5b199a5d, user_id=a390f532f2eb4da8af83a01d4397b6d8, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:54.856219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f58bc-db38-43ba-b604-6fe944b30b31 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:54.867036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 993/1959] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:29:54 2022] POST /identity/v3/users => generated 296 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:54.870588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:54.906856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 967/1960] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:54 2022] PUT /identity/v3/projects/fc5102d8b13d4212a2dbef0b5b199a5d/users/a390f532f2eb4da8af83a01d4397b6d8/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:54.924615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966790b9-96df-46e2-a47d-02a7777e0a2f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:54.926808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:54.927454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:54.929155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:54.992517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] MFA Rules not processed for user `a390f532f2eb4da8af83a01d4397b6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:54.993924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:55.031996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:55.109398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:55.110770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:55.111614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:55.112157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:55.115005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:55.118118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:55.137569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 968/1961] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:55.159483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:55.161762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:55.166623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 994/1962] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:29:55.172765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:55.194220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f926c-266e-4671-b7a9-c4706de2299e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:55.205551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:55.208539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:55.209449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:55.211367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] PATH_INFO: `/v3/projects/0f0818679c664927974008eb2f4dd578/users/b2d56d1db8154074b391f6aa414ef0ef/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:55.280213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0818679c664927974008eb2f4dd578, user_id=b2d56d1db8154074b391f6aa414ef0ef, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:55.296287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:55.312985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082d1f4-e2c5-468f-8bb3-3820ea70e324 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:55.330021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:55.330906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:55.331196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:55.331430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:55.356699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 995/1963] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:55 2022] PUT /identity/v3/projects/0f0818679c664927974008eb2f4dd578/users/b2d56d1db8154074b391f6aa414ef0ef/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:55.360891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:55.365277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:55.368793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:55.397663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 969/1964] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:55.540336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e29ed5-54f3-490d-8fce-55b2c93779a2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:55.551071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b3975-c49d-415e-97fe-54caaae20cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:55.553001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b3975-c49d-415e-97fe-54caaae20cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:55.554107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b3975-c49d-415e-97fe-54caaae20cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:55.554766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b3975-c49d-415e-97fe-54caaae20cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:55.556032 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b3975-c49d-415e-97fe-54caaae20cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:55.560615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b3975-c49d-415e-97fe-54caaae20cf3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:55.589476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 996/1965] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:29:55.596359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:55.619775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494b08-1037-408c-9ec9-ad2d4989b5f0 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:55.632159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:55.633742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:55.637265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:55.649450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] PATH_INFO: `/v3/projects/0f0818679c664927974008eb2f4dd578/users/b2d56d1db8154074b391f6aa414ef0ef/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:55.658049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0818679c664927974008eb2f4dd578, user_id=b2d56d1db8154074b391f6aa414ef0ef, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:55.678529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:55.743388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 970/1966] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:29:55 2022] PUT /identity/v3/projects/0f0818679c664927974008eb2f4dd578/users/b2d56d1db8154074b391f6aa414ef0ef/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:55.750570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:55.751381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:55.751751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:55.820344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] MFA Rules not processed for user `b2d56d1db8154074b391f6aa414ef0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:55.822964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:55.857858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:56.050151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:56.052936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:56.079959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 997/1967] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:56.106192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:56.114059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:56.237605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a4662-b17b-4e25-94e3-dd23cc1ee96f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:56.245964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:56.247121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:56.247482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:56.247832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] PATH_INFO: `/v3/users/d3fba333ea45449c854cbe632bc853d9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:56.249350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3fba333ea45449c854cbe632bc853d9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:56.258760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:56.266592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910124d-576b-48c8-8b6c-38756fa44e12 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:56.271012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2655d-d661-4ad0-a60b-42050890a37e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:56.271700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2655d-d661-4ad0-a60b-42050890a37e service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:56.271986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2655d-d661-4ad0-a60b-42050890a37e service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:56.272300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2655d-d661-4ad0-a60b-42050890a37e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:56.288600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2655d-d661-4ad0-a60b-42050890a37e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:56.292701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2655d-d661-4ad0-a60b-42050890a37e service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:56.309799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 998/1968] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:56.403966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3fba333ea45449c854cbe632bc853d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:56.407730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:56.425349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3fba333ea45449c854cbe632bc853d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:56.445676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3fba333ea45449c854cbe632bc853d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:56.475134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3fba333ea45449c854cbe632bc853d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:56.476074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3fba333ea45449c854cbe632bc853d9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:56.495142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 971/1969] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:56 2022] DELETE /identity/v3/users/d3fba333ea45449c854cbe632bc853d9 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:29:56.502316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:56.502692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:56.503084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:56.533786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:56.534077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:56.620534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 999/1970] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:56.629957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:56.633065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:56.758628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13be3-be59-4722-96a2-1b3ea3503141 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:56.766033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:56.767183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:56.767671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:56.768232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:56.786777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:56.789715 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:56.812099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 972/1971] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:29:56.862937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:56.864490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:56.955069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:56.977664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9910026-c29d-4e4f-be76-a98ca46adb32 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:56.987777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d56d1db8154074b391f6aa414ef0ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f0818679c664927974008eb2f4dd578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:56.989319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:56.989822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:56.990305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:56.991898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:56.992499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:56.996476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:57.027667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1000/1972] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:29:57.029808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:57.052674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a78f6-c527-45c8-bdc9-37cafe94a1f4 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:57.061599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d56d1db8154074b391f6aa414ef0ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f0818679c664927974008eb2f4dd578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:57.062788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:57.063279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:57.063775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:57.065244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:57.069008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:57.129104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 973/1973] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:29:57 2022] POST /identity/v3/projects => generated 353 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:29:57.136505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:57.148270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66951e9-08b7-4bc3-b586-ad95bf908417 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:57.279187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d56d1db8154074b391f6aa414ef0ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f0818679c664927974008eb2f4dd578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:57.280224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:57.280745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:57.281325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:57.282675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:29:57.283153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:57.285606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:57.310762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1001/1974] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:29:57.399059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:57.427034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:57.429889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:57.432747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fbac4-b01b-483c-a280-7fd5b5ecbab3 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:57.444248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d56d1db8154074b391f6aa414ef0ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f0818679c664927974008eb2f4dd578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:57.448151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:57.448401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:57.451106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] PATH_INFO: `/v3/projects/eda6e1776dc546fc8517d514a2d522d9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:57.459087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eda6e1776dc546fc8517d514a2d522d9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:57.473927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:57.624802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda6e1776dc546fc8517d514a2d522d9', 'request_id': 'req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:57.642635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda6e1776dc546fc8517d514a2d522d9', 'request_id': 'req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:57.676025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda6e1776dc546fc8517d514a2d522d9', 'request_id': 'req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:57.696964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802091a1-941a-4d34-9142-c7e32782bded tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:57.720413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] The token cache is being invalidate because project eda6e1776dc546fc8517d514a2d522d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:57.721040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:57.722679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:57.728226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 974/1975] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:57 2022] DELETE /identity/v3/projects/eda6e1776dc546fc8517d514a2d522d9 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:57.888094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:57.890048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:57.890048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:57.890048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] PATH_INFO: `/v3/projects/40897ba73b574fcb8c13a1410e6b8d44` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:57.891683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] RBAC: Authorizing `identity:delete_project(project_id=40897ba73b574fcb8c13a1410e6b8d44)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:57.894161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:57.992088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:58.066175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40897ba73b574fcb8c13a1410e6b8d44', 'request_id': 'req-599c9545-469c-4f80-b1bc-35b378f0b64f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.115175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40897ba73b574fcb8c13a1410e6b8d44', 'request_id': 'req-599c9545-469c-4f80-b1bc-35b378f0b64f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.156376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40897ba73b574fcb8c13a1410e6b8d44', 'request_id': 'req-599c9545-469c-4f80-b1bc-35b378f0b64f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.232031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:58.234376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:58.236754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] The token cache is being invalidate because project 40897ba73b574fcb8c13a1410e6b8d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:58.237374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.238622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.246518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1002/1976] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:57 2022] DELETE /identity/v3/projects/40897ba73b574fcb8c13a1410e6b8d44 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:58.248741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:58.251753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:58.294131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031ccd0-db12-48eb-95dd-c1a84cb3d8d5 tempest-VolumeQuotaClassesTest-927930145 tempest-VolumeQuotaClassesTest-927930145-project-admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:58.374602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:58.467351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:58.469415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:58.469993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:58.470560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] PATH_INFO: `/v3/users/ea19e35727714e7f98170a6a34a64a54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:58.476505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea19e35727714e7f98170a6a34a64a54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:58.484296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:58.505972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c9545-469c-4f80-b1bc-35b378f0b64f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:58.513499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:58.514819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:58.514819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:58.514975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] PATH_INFO: `/v3/users/4f927d9891194d7b9b0f3eb769b5dd73` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:58.517279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f927d9891194d7b9b0f3eb769b5dd73)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:58.519480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:58.608477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea19e35727714e7f98170a6a34a64a54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.609145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f927d9891194d7b9b0f3eb769b5dd73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.623554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea19e35727714e7f98170a6a34a64a54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.628015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f927d9891194d7b9b0f3eb769b5dd73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.640177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea19e35727714e7f98170a6a34a64a54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.643736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f927d9891194d7b9b0f3eb769b5dd73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.645687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f927d9891194d7b9b0f3eb769b5dd73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.646608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e96b526b-1944-49ac-b6b7-f541177686c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f927d9891194d7b9b0f3eb769b5dd73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.660757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea19e35727714e7f98170a6a34a64a54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.661554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea19e35727714e7f98170a6a34a64a54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:58.671436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1003/1977] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:58 2022] DELETE /identity/v3/users/4f927d9891194d7b9b0f3eb769b5dd73 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:58.676310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 975/1978] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:58 2022] DELETE /identity/v3/users/ea19e35727714e7f98170a6a34a64a54 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:58.679207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:58.679742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:58.680282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:58.709791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:58.712134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:58.776438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:58.823902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1004/1979] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:29:58.843001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:58.846302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:58.964553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59ac03-5a9c-4cb1-8644-b49ed030f4e0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:58.971251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:58.972304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:58.972772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:58.973248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:58.999613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:59.002927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:59.024118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 976/1980] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:29:59.238994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:59.247459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:59.328380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69234b7c-5d09-4d47-8e9c-0ac8286fe84b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:59.343903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:59.346460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:59.346930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:59.347412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] PATH_INFO: `/v3/projects/80ad90bc11be4eb9b52a65e548422aca` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:59.352886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ad90bc11be4eb9b52a65e548422aca)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:59.358470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:59.505025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ad90bc11be4eb9b52a65e548422aca', 'request_id': 'req-0025901a-d08c-413f-b7d8-004fc3ce241b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.545024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ad90bc11be4eb9b52a65e548422aca', 'request_id': 'req-0025901a-d08c-413f-b7d8-004fc3ce241b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.569556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ad90bc11be4eb9b52a65e548422aca', 'request_id': 'req-0025901a-d08c-413f-b7d8-004fc3ce241b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.609204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] The token cache is being invalidate because project 80ad90bc11be4eb9b52a65e548422aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:29:59.609784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.610985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0025901a-d08c-413f-b7d8-004fc3ce241b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.613994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1005/1981] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:29:59 2022] DELETE /identity/v3/projects/80ad90bc11be4eb9b52a65e548422aca => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:29:59.620700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:59.638599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:59.639106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:59.640073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:59.646161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4cbeb-345a-4340-8497-c918f1daa434 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:29:59.683735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:29:59.686977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:59.763492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:29:59.788994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:29:59.789748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:29:59.793308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:29:59.796768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] PATH_INFO: `/v3/users/14a5711bb1ab4e6d80b207732a98c764` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:29:59.800850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a5711bb1ab4e6d80b207732a98c764)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:29:59.803807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:29:59.842143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1006/1982] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:29:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:29:59.865650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:29:59.868651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:29:59.875695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a5711bb1ab4e6d80b207732a98c764'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.903862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a5711bb1ab4e6d80b207732a98c764'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.918081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a5711bb1ab4e6d80b207732a98c764'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.942783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a5711bb1ab4e6d80b207732a98c764'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.943657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a5711bb1ab4e6d80b207732a98c764'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:29:59.958375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 977/1983] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:29:59 2022] DELETE /identity/v3/users/14a5711bb1ab4e6d80b207732a98c764 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:00.011776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:00.014549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:00.124994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7a671-9762-4789-ba0d-394fe7ca7546 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:00.138346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:00.185657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d3a18-a047-478d-a7ec-8c1b5fa84dfe admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:00.194990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:00.196599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:00.197183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:00.197568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] PATH_INFO: `/v3/projects/bfdd3a05b6154d98a34c6c5c4453be2d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:00.199799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfdd3a05b6154d98a34c6c5c4453be2d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:00.202968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:00.296604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:00.299982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:00.301059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:00.301559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:00.305324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:00.321537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfdd3a05b6154d98a34c6c5c4453be2d', 'request_id': 'req-364298ca-df4f-41b4-bbe5-28ca17350e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.335512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfdd3a05b6154d98a34c6c5c4453be2d', 'request_id': 'req-364298ca-df4f-41b4-bbe5-28ca17350e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.354794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfdd3a05b6154d98a34c6c5c4453be2d', 'request_id': 'req-364298ca-df4f-41b4-bbe5-28ca17350e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.396155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] The token cache is being invalidate because project bfdd3a05b6154d98a34c6c5c4453be2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:00.396746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.398780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.406149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 978/1984] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:00 2022] DELETE /identity/v3/projects/bfdd3a05b6154d98a34c6c5c4453be2d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:00.413603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:00.416491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:00.427299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:00.429002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:00.429699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:00.441869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1007/1985] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:29:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:30:00.515177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:00.515177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:00.552203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83687382-b8ee-4304-b665-3c6d542d14e1 service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:00.601290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364298ca-df4f-41b4-bbe5-28ca17350e5e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:00.608999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:00.610117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:00.610364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:00.612841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] PATH_INFO: `/v3/users/b2d56d1db8154074b391f6aa414ef0ef` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:00.615684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d56d1db8154074b391f6aa414ef0ef)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:00.620944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:00.670355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:00.671910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:00.672481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:00.673139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] PATH_INFO: `/v3/projects/31a99c42b19a46e2805cb68137ea50ec` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:00.674957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a99c42b19a46e2805cb68137ea50ec)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:00.677994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:00.712304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d56d1db8154074b391f6aa414ef0ef'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.730492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d56d1db8154074b391f6aa414ef0ef'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.741714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d56d1db8154074b391f6aa414ef0ef'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.746975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a99c42b19a46e2805cb68137ea50ec', 'request_id': 'req-75118720-8122-42ad-a277-a71c0ddf7d5d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.749929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:00.757761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a99c42b19a46e2805cb68137ea50ec', 'request_id': 'req-75118720-8122-42ad-a277-a71c0ddf7d5d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.766466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a99c42b19a46e2805cb68137ea50ec', 'request_id': 'req-75118720-8122-42ad-a277-a71c0ddf7d5d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.766717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d56d1db8154074b391f6aa414ef0ef'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.767423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-facdac6f-fb54-48db-b8c8-64e643b2421c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d56d1db8154074b391f6aa414ef0ef'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.783815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 979/1986] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:00 2022] DELETE /identity/v3/users/b2d56d1db8154074b391f6aa414ef0ef => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:00.799711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] The token cache is being invalidate because project 31a99c42b19a46e2805cb68137ea50ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:00.800357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.801534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-75118720-8122-42ad-a277-a71c0ddf7d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:00.804385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1008/1987] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:00 2022] DELETE /identity/v3/projects/31a99c42b19a46e2805cb68137ea50ec => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:00.811498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:00.811812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:00.812117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:00.841110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:00.842610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:00.951929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 980/1988] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:00.960810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:00.961096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:00.961368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.004479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:01.007837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.033455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1009/1989] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:01.038223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.070904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480a3d1-7745-4425-a1b2-295572e99489 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.081545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.082935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.084037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.084570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.086481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:01.086868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.090879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.124152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 981/1990] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:01.129763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.158739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715e820-ab49-4251-8df2-7d1f71bfa11e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.172265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a3a9d-c8ca-4842-8cf6-e99672e5430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.173464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a3a9d-c8ca-4842-8cf6-e99672e5430d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.173971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a3a9d-c8ca-4842-8cf6-e99672e5430d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.174529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a3a9d-c8ca-4842-8cf6-e99672e5430d admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.176127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a3a9d-c8ca-4842-8cf6-e99672e5430d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.179865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a3a9d-c8ca-4842-8cf6-e99672e5430d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.237748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1010/1991] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:01 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:01.243841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.265453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf73a57-68fb-442f-adf7-2fa931e4f4cb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.277675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:01.464680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.465200 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.465805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.469074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.470106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.470453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.470786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.471701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.475038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.510102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 982/1992] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:01 2022] POST /identity/v3/users => generated 284 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:01.511912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.514046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.529862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:01.531680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.536610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e4349-b0c8-4ff1-87b4-3c25212c73d5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.545895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.547236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.547613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.548822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] PATH_INFO: `/v3/projects/0f0818679c664927974008eb2f4dd578` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.551327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0818679c664927974008eb2f4dd578)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.553127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.633415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1011/1993] 10.222.0.30 () {58 vars in 967 bytes} [Thu Jun 16 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:01.634350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.639784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0818679c664927974008eb2f4dd578', 'request_id': 'req-aae6c681-9ec0-4bfa-80d4-77f97959628f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:01.651438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0818679c664927974008eb2f4dd578', 'request_id': 'req-aae6c681-9ec0-4bfa-80d4-77f97959628f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:01.661045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0818679c664927974008eb2f4dd578', 'request_id': 'req-aae6c681-9ec0-4bfa-80d4-77f97959628f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:01.665748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd463f-0dfe-47fa-a1f6-5f5c56e9936b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.682106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] The token cache is being invalidate because project 0f0818679c664927974008eb2f4dd578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:01.682409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:01.683291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:01.685168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 983/1994] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:01 2022] DELETE /identity/v3/projects/0f0818679c664927974008eb2f4dd578 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:01.686200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.688160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.798480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6c681-9ec0-4bfa-80d4-77f97959628f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.804202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7999446-6988-436b-bb4e-a012f6ab0a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.805635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7999446-6988-436b-bb4e-a012f6ab0a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.806188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7999446-6988-436b-bb4e-a012f6ab0a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.807532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7999446-6988-436b-bb4e-a012f6ab0a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.809166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7999446-6988-436b-bb4e-a012f6ab0a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.809442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.810225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.810583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.810937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.813237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7999446-6988-436b-bb4e-a012f6ab0a42 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.813644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.848505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1012/1995] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:01.848776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.850367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.850978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.858188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:01.899667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:01.902766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.913609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1013/1996] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:01.916342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:01.918722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.922106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.924086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5880e0-2e8c-4777-8dd8-4ab0373eb640 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.948404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 984/1997] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:30:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:30:01.950160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.950580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.950840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.954651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8b77-b7b8-43c0-a6d9-dc56eee619cc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:01.970493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:01.971858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:01.972501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:01.972874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:01.974495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:01.976854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:01.979488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:01.981498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:01.991842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 985/1998] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:01.997353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.000127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1014/1999] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:02.003965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.012396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f01a56-a8d9-4a81-a17f-e4e445fde7d0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.021476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.021876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.022781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.023027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.024720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:02.026231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.029812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.052434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e660da-6ba2-4e24-9b7a-fac3c8a3726d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.067411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 986/2000] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:02.067965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.069491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.069971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.070444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] PATH_INFO: `/v3/projects/6d91db54b9c2472eb44f90c7d6f4c411/users/3287fc466d684c3a8444493f2ff12aee/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.071888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.085452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2e6e8-7575-4e81-ab52-dd58419a5a70 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.091583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdf9fa-91c9-48bf-a8f5-e67cd3176668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.092297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf9fa-91c9-48bf-a8f5-e67cd3176668 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.092578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf9fa-91c9-48bf-a8f5-e67cd3176668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.092848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf9fa-91c9-48bf-a8f5-e67cd3176668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.093566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdf9fa-91c9-48bf-a8f5-e67cd3176668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.095566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdf9fa-91c9-48bf-a8f5-e67cd3176668 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.097935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d91db54b9c2472eb44f90c7d6f4c411, user_id=3287fc466d684c3a8444493f2ff12aee, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.103057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.126182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1015/2001] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:02 2022] PUT /identity/v3/projects/6d91db54b9c2472eb44f90c7d6f4c411/users/3287fc466d684c3a8444493f2ff12aee/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:02.131429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 987/2002] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:02 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:02.137301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.138163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.138163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.138163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.159857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:02.179651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] MFA Rules not processed for user `3287fc466d684c3a8444493f2ff12aee`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:02.181237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:02.255249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e110a3-3b9b-4382-a374-cb97abc1c478 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.284742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 988/2003] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:02.305461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.309517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:02.336448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09256b86-5b1e-4866-b757-1e2173fec59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.337049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09256b86-5b1e-4866-b757-1e2173fec59b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.337369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09256b86-5b1e-4866-b757-1e2173fec59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.337600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09256b86-5b1e-4866-b757-1e2173fec59b admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.339278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09256b86-5b1e-4866-b757-1e2173fec59b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.341577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09256b86-5b1e-4866-b757-1e2173fec59b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.368178 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1016/2004] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:02 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:02.369935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.370448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.370932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.401554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:02.405215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:02.408948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf2369-16bd-4717-a9b3-7d531b533dcd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.412976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.413624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.414045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.424492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.446319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.448650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.460971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 989/2005] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:02.475413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.490222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e4fd-b005-45e3-86c9-956a2d4f69b9 service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.498382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.499692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.500202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.500557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.502119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.503742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.536086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 990/2006] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:02.542734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.551598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:02.562304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f9cf-7e0b-485d-b54b-c0976b4153e4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.588264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.590193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.590810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.591348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.593033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.596538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.628525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 991/2007] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:02.634607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1017/2008] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:30:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:02.643473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.668437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754e4a3-d8c4-42e4-b81d-c84dbbaa9534 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.680564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bb032-f36c-4b6f-b07d-88aaf224c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:02.682113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:02.683152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bb032-f36c-4b6f-b07d-88aaf224c91a admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.683615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bb032-f36c-4b6f-b07d-88aaf224c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.684114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bb032-f36c-4b6f-b07d-88aaf224c91a admin admin] PATH_INFO: `/v3/projects/919e38fad0bd47018dd71bb195499827/users/a228888d552e480c8b9cbdc9ff374543/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.732256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bb032-f36c-4b6f-b07d-88aaf224c91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=919e38fad0bd47018dd71bb195499827, user_id=a228888d552e480c8b9cbdc9ff374543, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:02.735864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bb032-f36c-4b6f-b07d-88aaf224c91a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:02.761297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 992/2009] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:02 2022] PUT /identity/v3/projects/919e38fad0bd47018dd71bb195499827/users/a228888d552e480c8b9cbdc9ff374543/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:02.774711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:02.777581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:02.778278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:02.809372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:02.833497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fcf8f-4da6-4f2c-8949-722eaf835215 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:02.896448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] MFA Rules not processed for user `a228888d552e480c8b9cbdc9ff374543`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:02.896448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:03.022478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:03.023900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:03.025601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:03.026339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:03.139818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 993/2010] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:03.141618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:03.185031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:03.192395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:03.231232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1018/2011] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:30:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:30:03.320614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f3d5-44f7-4f20-bb26-ce4576a11700 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:03.340506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:03.340506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:03.340506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:03.340506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:03.341029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:03.345056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:03.445621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 994/2012] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:03 2022] POST /identity/v3/projects => generated 401 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:03.453706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:03.488172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf4539-0ffe-4c1a-9710-c6d70065fffa service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:03.661130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:03.662119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:03.662119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:03.662538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:03.663425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:03.666859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:03.706870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1019/2013] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:03 2022] POST /identity/v3/users => generated 307 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:03.715221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:03.732898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff55bbb-ad3f-4736-b66a-884c48978741 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:03.742595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:03.743315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:03.744089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:03.747481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:03.752515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:03.755625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:03.771970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 995/2014] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:03.778773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:03.801012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f6d0f-3d2d-49f1-bd0f-bb6337b451fa admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:03.810839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31db226-b9d0-47d9-b3a9-e49d8297fd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:03.812360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31db226-b9d0-47d9-b3a9-e49d8297fd0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:03.812963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31db226-b9d0-47d9-b3a9-e49d8297fd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:03.813824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31db226-b9d0-47d9-b3a9-e49d8297fd0d admin admin] PATH_INFO: `/v3/projects/d92c607765a34a9ea0750cdcc36f7a95/users/5c74f50b44014248986c512b45f986fd/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:03.855857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31db226-b9d0-47d9-b3a9-e49d8297fd0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92c607765a34a9ea0750cdcc36f7a95, user_id=5c74f50b44014248986c512b45f986fd, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:03.861507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31db226-b9d0-47d9-b3a9-e49d8297fd0d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:03.886730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1020/2015] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:03 2022] PUT /identity/v3/projects/d92c607765a34a9ea0750cdcc36f7a95/users/5c74f50b44014248986c512b45f986fd/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:03.891668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:03.904586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ddab-32f9-42fc-99a9-baf1787e9b72 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:03.949393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:04.027565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:04.028430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.028915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.029505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.030847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:04.034512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:04.050805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.051099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.051786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.063624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:04.067543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 996/2016] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:04.074921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:04.091153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9d74-c5d3-4c06-9aef-8e3d4782f530 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:04.099081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:04.099549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12187d1-bc80-4ed0-ab40-6caf53e225a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:04.100535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12187d1-bc80-4ed0-ab40-6caf53e225a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.100803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12187d1-bc80-4ed0-ab40-6caf53e225a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.101089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12187d1-bc80-4ed0-ab40-6caf53e225a8 admin admin] PATH_INFO: `/v3/projects/d92c607765a34a9ea0750cdcc36f7a95/users/5c74f50b44014248986c512b45f986fd/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.102244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:04.104786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12187d1-bc80-4ed0-ab40-6caf53e225a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92c607765a34a9ea0750cdcc36f7a95, user_id=5c74f50b44014248986c512b45f986fd, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:04.109626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12187d1-bc80-4ed0-ab40-6caf53e225a8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:04.126114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 997/2017] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:04 2022] PUT /identity/v3/projects/d92c607765a34a9ea0750cdcc36f7a95/users/5c74f50b44014248986c512b45f986fd/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:04.145738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.146111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.146647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.188832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] MFA Rules not processed for user `5c74f50b44014248986c512b45f986fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:04.192089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:04.233214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1021/2018] 10.222.0.30 () {58 vars in 967 bytes} [Thu Jun 16 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:04.249482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:04.300639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 998/2019] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:04.317408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:04.326844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:04.398181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99894d20-20ed-4243-80b6-61c26dd77edd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:04.448280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2d22d-64a3-4197-ac46-32c24048ad01 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:04.454114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:04.454822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.455116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.455401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.468871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:04.471685 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:04.498744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 999/2020] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:04.524487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:04.525965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.526385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.526668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.610063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:04.627584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:04.629698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:04.631491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:04.646172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13cf7-850a-409c-845e-b3401ff87283 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:04.652242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1022/2021] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:30:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:30:04.653758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960fa97-1f2c-4c29-b097-09a53a38e7fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:04.654618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960fa97-1f2c-4c29-b097-09a53a38e7fd service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:04.655277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960fa97-1f2c-4c29-b097-09a53a38e7fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:04.655562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960fa97-1f2c-4c29-b097-09a53a38e7fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:04.737020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:04.770687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960fa97-1f2c-4c29-b097-09a53a38e7fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:04.774088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960fa97-1f2c-4c29-b097-09a53a38e7fd service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:04.788605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1000/2022] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:06.699906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:06.704626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:06.858880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da378acb-6998-48f6-b314-9e1b661b44e5 service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:06.868078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:06.869653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:06.870361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:06.870672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] PATH_INFO: `/v3/users/a390f532f2eb4da8af83a01d4397b6d8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:06.873102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a390f532f2eb4da8af83a01d4397b6d8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:06.874526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:07.017774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a390f532f2eb4da8af83a01d4397b6d8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.108036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a390f532f2eb4da8af83a01d4397b6d8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.141576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a390f532f2eb4da8af83a01d4397b6d8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.143601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a390f532f2eb4da8af83a01d4397b6d8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.146150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a390f532f2eb4da8af83a01d4397b6d8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.195945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1023/2023] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:06 2022] DELETE /identity/v3/users/a390f532f2eb4da8af83a01d4397b6d8 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:07.208749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:07.209657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:07.209657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:07.251016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:07.257230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:07.370986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1001/2024] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:07.390242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:07.392177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:07.492388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e60b-0ab2-4b02-84d8-87ea9290d585 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:07.499455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8d532-2ca9-4fce-ba9a-0b026af90ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:07.500630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8d532-2ca9-4fce-ba9a-0b026af90ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:07.501194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8d532-2ca9-4fce-ba9a-0b026af90ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:07.501815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8d532-2ca9-4fce-ba9a-0b026af90ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:07.522333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8d532-2ca9-4fce-ba9a-0b026af90ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:07.526436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8d532-2ca9-4fce-ba9a-0b026af90ea9 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:07.544565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1024/2025] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:07.724602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:07.726408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:07.748095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c1190-17c4-4460-aa74-f79ac4c42949 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:07.759187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:07.762606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:07.763065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:07.763527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] PATH_INFO: `/v3/users/479bda47e66544e8bfa47dad8294c180` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:07.766438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=479bda47e66544e8bfa47dad8294c180)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:07.770362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:07.920961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '479bda47e66544e8bfa47dad8294c180'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.938455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '479bda47e66544e8bfa47dad8294c180'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:07.951900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '479bda47e66544e8bfa47dad8294c180'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.001885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '479bda47e66544e8bfa47dad8294c180'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.003134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '479bda47e66544e8bfa47dad8294c180'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.017873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1002/2026] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:07 2022] DELETE /identity/v3/users/479bda47e66544e8bfa47dad8294c180 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:08.022554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:08.023533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:08.023533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:08.050691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:08.052030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:08.072559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:08.141006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:08.142747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:08.297358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1025/2027] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:08.312685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:08.350909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:08.366334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:08.398234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f0042-c5eb-4d59-ba7e-e0c71445fbc1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:08.417205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:08.418574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:08.418864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:08.419321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] PATH_INFO: `/v3/projects/fc5102d8b13d4212a2dbef0b5b199a5d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:08.422914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5102d8b13d4212a2dbef0b5b199a5d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:08.424983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:08.622538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf4fe6-d9d1-4b81-9cf7-e01af84e2b50 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:08.641689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5102d8b13d4212a2dbef0b5b199a5d', 'request_id': 'req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.669646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5102d8b13d4212a2dbef0b5b199a5d', 'request_id': 'req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.691049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5102d8b13d4212a2dbef0b5b199a5d', 'request_id': 'req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.735283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] The token cache is being invalidate because project fc5102d8b13d4212a2dbef0b5b199a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:08.735582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.744359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad007d44-c76a-43b3-b2d4-a4d4454d0c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:08.747709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1003/2028] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:08 2022] DELETE /identity/v3/projects/fc5102d8b13d4212a2dbef0b5b199a5d => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:08.757773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01cffd-09c3-447c-8069-3e8f54697bd2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:08.758362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01cffd-09c3-447c-8069-3e8f54697bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:08.758559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01cffd-09c3-447c-8069-3e8f54697bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:08.802589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6f01cffd-09c3-447c-8069-3e8f54697bd2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:08.803955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6f01cffd-09c3-447c-8069-3e8f54697bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:08.849979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:08.851140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:08.851646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:08.852578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:08.862878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:09.006331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1004/2029] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:09.008938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.009354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.010231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.035977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:09.037402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:09.038442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.040075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.054928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1026/2031] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:09.056396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1005/2031] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:09.069559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.095446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3adca2-5804-4251-bc4f-c20ac884dc59 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.106788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.107978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.108651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.109380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.110713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:09.111263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.114662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.145416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1027/2032] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:09.151655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.180961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61570-1fc9-45c7-bd4a-732578be6643 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.201792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.203211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.204032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.204928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.206370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.211016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.283680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1006/2033] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:09 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:09.296298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.327303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14335001-a1c2-44ee-bbe6-6782298c4c5a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.340641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:09.349085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.351937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:09.505964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:09.516035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad33206-0c2b-41f8-b4d9-b740191edeb2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.523148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.523852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.524181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.524510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.525371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.527597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.528006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.529994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.532688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.533382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] PATH_INFO: `/v3/projects/478dbeee61b6401c9dbb31c216ce537b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.535562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478dbeee61b6401c9dbb31c216ce537b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.540375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.557377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1028/2034] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:09 2022] POST /identity/v3/users => generated 282 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:09.568257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.583102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfab81-9f94-4fd0-b879-6f75aefde32d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.589686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.590442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.590734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.591282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.591985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.593803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.612194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1029/2035] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:09.613695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.626848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c323a2-5c7d-4a84-ba48-c0a3df36b197 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.644796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478dbeee61b6401c9dbb31c216ce537b', 'request_id': 'req-b1b402b8-3bee-427d-94af-155055d9a4d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:09.657056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478dbeee61b6401c9dbb31c216ce537b', 'request_id': 'req-b1b402b8-3bee-427d-94af-155055d9a4d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:09.670611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478dbeee61b6401c9dbb31c216ce537b', 'request_id': 'req-b1b402b8-3bee-427d-94af-155055d9a4d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:09.725689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] The token cache is being invalidate because project 478dbeee61b6401c9dbb31c216ce537b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:09.725945 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:09.726735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b1b402b8-3bee-427d-94af-155055d9a4d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:09.729264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1007/2036] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:09 2022] DELETE /identity/v3/projects/478dbeee61b6401c9dbb31c216ce537b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:09.736666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c23afd-f73c-4f75-8fdf-8616eda285c7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.737220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c23afd-f73c-4f75-8fdf-8616eda285c7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.737734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c23afd-f73c-4f75-8fdf-8616eda285c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.767116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.768176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.768469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.768752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.769581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.771456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.772350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-07c23afd-f73c-4f75-8fdf-8616eda285c7 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:09.774943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07c23afd-f73c-4f75-8fdf-8616eda285c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:09.800899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1030/2037] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:09.802737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.809160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:09.828126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1008/2038] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:09.831844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab5423-1f33-4dbf-a795-a67bd5165bd5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.834143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.834961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.835258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.842283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6300ac5-7828-4348-9645-8288f1b684b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.843570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6300ac5-7828-4348-9645-8288f1b684b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.843956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6300ac5-7828-4348-9645-8288f1b684b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.844362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6300ac5-7828-4348-9645-8288f1b684b4 admin admin] PATH_INFO: `/v3/projects/41f24044f5f84ce5b3653519d325a92a/users/bbb32a220826411487abab6cb4bf9f86/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.865171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:09.869917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:09.887744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1009/2039] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:09.889343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:09.938882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1dd25-2765-4d1d-bfee-6be2387e2d51 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:09.939255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6300ac5-7828-4348-9645-8288f1b684b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f24044f5f84ce5b3653519d325a92a, user_id=bbb32a220826411487abab6cb4bf9f86, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.951690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:09.953137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:09.953665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:09.954446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6300ac5-7828-4348-9645-8288f1b684b4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.954747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:09.955755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:09.956595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:09.958880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:09.990584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1010/2040] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:09.992370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:10.008267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4b94b-df10-4b26-9e3b-88f06eeb8692 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:10.032221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1031/2041] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:09 2022] PUT /identity/v3/projects/41f24044f5f84ce5b3653519d325a92a/users/bbb32a220826411487abab6cb4bf9f86/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:10.065243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:10.066056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:10.066149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:10.170424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] MFA Rules not processed for user `bbb32a220826411487abab6cb4bf9f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:10.172940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:10.224321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:10.225518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:10.225964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:10.226351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:10.227482 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:10.240012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:10.339046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1011/2042] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:09 2022] POST /identity/v3/projects => generated 385 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:10.349273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:10.401908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:10.460198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1032/2043] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:10.474472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:10.477113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:10.543941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:10.603313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36ca17-e30b-46cb-9416-04a2677ef387 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:10.615240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:10.616418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:10.616911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:10.617378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:10.618349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:10.622135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:10.631945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaf538-2809-4ceb-a775-c73612495366 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:10.675179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1012/2044] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:10 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:10.675179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:10.677754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:10.705462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acd600-8aa2-4111-a4a6-00a7be89e519 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:10.715208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:10.716861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:10.717633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:10.718164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] PATH_INFO: `/v3/users/a228888d552e480c8b9cbdc9ff374543` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:10.720597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a228888d552e480c8b9cbdc9ff374543)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:10.724890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:10.736551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:10.737492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:10.737987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:10.738466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:10.827668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a228888d552e480c8b9cbdc9ff374543'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:10.836541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a228888d552e480c8b9cbdc9ff374543'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:10.847871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:10.849917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:10.856943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a228888d552e480c8b9cbdc9ff374543'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:10.869976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1033/2045] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:10.871723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:10.881205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a228888d552e480c8b9cbdc9ff374543'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:10.882942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d853dc7f-7652-420b-bb44-0483430bfbc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a228888d552e480c8b9cbdc9ff374543'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:10.892744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1013/2046] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:10 2022] DELETE /identity/v3/users/a228888d552e480c8b9cbdc9ff374543 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:10.899032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:10.899360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:10.899623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:10.900709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf40f6-1030-49e5-b0df-2bd22710bffe service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:10.929302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:10.930886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:11.031092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:11.051022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:11.052081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:11.052654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:11.053078 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:11.054454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:11.067328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1014/2047] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:11.072531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:11.081604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:11.083537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:11.098064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1034/2048] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:11.099917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:11.104035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:11.156821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3048f5-6f01-4b43-9420-3f9637ad5da7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:11.174412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:11.175219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:11.175535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:11.175773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:11.183543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:11.183543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:11.216526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1035/2049] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:11.219447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:11.231479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:11.239430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933e5af-42f6-4e58-8344-b6c45c7a6acb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:11.247842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e33d52-788c-4837-ac9b-73c7a006b3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:11.248953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e33d52-788c-4837-ac9b-73c7a006b3cc service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:11.249481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e33d52-788c-4837-ac9b-73c7a006b3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:11.249908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e33d52-788c-4837-ac9b-73c7a006b3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:11.255177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0452f3-59f7-4f18-b982-1deb33ea0ef3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:11.277020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e33d52-788c-4837-ac9b-73c7a006b3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:11.281985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e33d52-788c-4837-ac9b-73c7a006b3cc service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:11.285148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:11.286352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:11.286640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:11.286916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] PATH_INFO: `/v3/projects/c9fa2caa1412451aa0308c807d5c62d8/users/0d7d57c4231b49b28a606a17e42267e7/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:11.322263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1015/2050] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:11.383665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fa2caa1412451aa0308c807d5c62d8, user_id=0d7d57c4231b49b28a606a17e42267e7, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:11.399453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:11.436189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1036/2051] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:11 2022] PUT /identity/v3/projects/c9fa2caa1412451aa0308c807d5c62d8/users/0d7d57c4231b49b28a606a17e42267e7/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:11.450676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ec563-b743-4046-8f72-955bb875741e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:11.451779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ec563-b743-4046-8f72-955bb875741e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:11.452858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ec563-b743-4046-8f72-955bb875741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:11.497649 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-226ec563-b743-4046-8f72-955bb875741e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:11.499024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-226ec563-b743-4046-8f72-955bb875741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:11.746485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1016/2052] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:11.780819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:11.892034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178b112-684b-41ad-8ed3-acd08439d99a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:11.897627 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:11.898557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:11.898859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:11.899131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:11.913542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:11.915316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:11.939493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1037/2053] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:12.133753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ec563-b743-4046-8f72-955bb875741e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:12.136460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-226ec563-b743-4046-8f72-955bb875741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:12.159563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ec563-b743-4046-8f72-955bb875741e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:12.178077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:12.179637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:12.180162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:12.180642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] PATH_INFO: `/v3/projects/919e38fad0bd47018dd71bb195499827` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:12.185586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] RBAC: Authorizing `identity:delete_project(project_id=919e38fad0bd47018dd71bb195499827)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:12.197672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:12.311732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919e38fad0bd47018dd71bb195499827', 'request_id': 'req-4bf50e3d-98b3-463f-af40-59617557f95f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.320124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919e38fad0bd47018dd71bb195499827', 'request_id': 'req-4bf50e3d-98b3-463f-af40-59617557f95f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.329725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919e38fad0bd47018dd71bb195499827', 'request_id': 'req-4bf50e3d-98b3-463f-af40-59617557f95f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.347211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] The token cache is being invalidate because project 919e38fad0bd47018dd71bb195499827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:12.347561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.348624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4bf50e3d-98b3-463f-af40-59617557f95f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.352119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1017/2054] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:12 2022] DELETE /identity/v3/projects/919e38fad0bd47018dd71bb195499827 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:12.355749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:12.388173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6033a5-12f7-47d6-a9bc-78c3d2bea309 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:12.535831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:12.537808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:12.538381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:12.538847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] PATH_INFO: `/v3/users/5c74f50b44014248986c512b45f986fd` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:12.541573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c74f50b44014248986c512b45f986fd)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:12.546446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:12.654879 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c74f50b44014248986c512b45f986fd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.702973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:12.708444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c74f50b44014248986c512b45f986fd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.734545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c74f50b44014248986c512b45f986fd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.736029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c74f50b44014248986c512b45f986fd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.738657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5466c6ce-1fe6-4d92-8fd4-299ee58844f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c74f50b44014248986c512b45f986fd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:12.792235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1038/2055] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:12 2022] DELETE /identity/v3/users/5c74f50b44014248986c512b45f986fd => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:12.842331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf24722-3554-4b01-8362-d2314d1046a8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:12.844025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf24722-3554-4b01-8362-d2314d1046a8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:12.844593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf24722-3554-4b01-8362-d2314d1046a8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:12.848336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1018/2056] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:30:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:30:12.907526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:12.908028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:12.908558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:12.962218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:12.964593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:13.126047 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1039/2057] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:13.193718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf24722-3554-4b01-8362-d2314d1046a8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:13.195610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cdf24722-3554-4b01-8362-d2314d1046a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:13.227474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf24722-3554-4b01-8362-d2314d1046a8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:13.239606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:13.240324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:13.240832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:13.242895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:13.248529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:13.250106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:13.292281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:13.305542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:13.341670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1019/2058] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:13.442373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db781fb9-8179-4956-82b6-66a53ffae52e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:13.454018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:13.456040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:13.456555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:13.457316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] PATH_INFO: `/v3/users/7ee9c06c774f478db33c16b784d6b914` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:13.460529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee9c06c774f478db33c16b784d6b914)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:13.464855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:13.599385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:13.602583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:13.638545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee9c06c774f478db33c16b784d6b914'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.656801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884090-aa88-4780-8095-182e1c2c4900 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:13.673363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee9c06c774f478db33c16b784d6b914'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.694115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:13.694115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:13.694115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:13.694115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] PATH_INFO: `/v3/projects/d92c607765a34a9ea0750cdcc36f7a95` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:13.694970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92c607765a34a9ea0750cdcc36f7a95)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:13.696103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:13.723436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee9c06c774f478db33c16b784d6b914'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.724840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee9c06c774f478db33c16b784d6b914'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.725784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d5eecd05-6791-4489-9cf1-93b600e8bc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee9c06c774f478db33c16b784d6b914'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.734455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:13.788392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1040/2059] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:13 2022] DELETE /identity/v3/users/7ee9c06c774f478db33c16b784d6b914 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:13.897255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92c607765a34a9ea0750cdcc36f7a95', 'request_id': 'req-608fcbf9-19a7-4b29-9a0a-643c744c722f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.903774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:13.909512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92c607765a34a9ea0750cdcc36f7a95', 'request_id': 'req-608fcbf9-19a7-4b29-9a0a-643c744c722f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.924754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92c607765a34a9ea0750cdcc36f7a95', 'request_id': 'req-608fcbf9-19a7-4b29-9a0a-643c744c722f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.962045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] The token cache is being invalidate because project d92c607765a34a9ea0750cdcc36f7a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:13.962613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.964198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:13.966758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1020/2060] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:13 2022] DELETE /identity/v3/projects/d92c607765a34a9ea0750cdcc36f7a95 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:13.973836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b7a7c-9aef-4cb3-8252-b93b9e1cf4fd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:13.974476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b7a7c-9aef-4cb3-8252-b93b9e1cf4fd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:13.975104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b7a7c-9aef-4cb3-8252-b93b9e1cf4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.069797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7e1b7a7c-9aef-4cb3-8252-b93b9e1cf4fd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:14.074411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7e1b7a7c-9aef-4cb3-8252-b93b9e1cf4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:14.286069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1041/2061] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:14.297922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:14.301915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:14.316436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.318111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.318482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.332902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fcbf9-19a7-4b29-9a0a-643c744c722f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:14.338893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:14.340378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.340843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.341286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] PATH_INFO: `/v3/projects/aa9111175cad44b2bbc611b7ae36a991` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.344254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9111175cad44b2bbc611b7ae36a991)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:14.346813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:14.377642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:14.380398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:14.402955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1042/2062] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:14.409595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:14.425095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9111175cad44b2bbc611b7ae36a991', 'request_id': 'req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:14.434141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9111175cad44b2bbc611b7ae36a991', 'request_id': 'req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:14.444219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9111175cad44b2bbc611b7ae36a991', 'request_id': 'req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:14.460560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8d1a-184b-421c-91dc-ede1c3b1d7c0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:14.472333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] The token cache is being invalidate because project aa9111175cad44b2bbc611b7ae36a991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:14.473374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:14.481673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-783e06f6-6b0e-4934-87f2-f17dfb8f1ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:14.487905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1021/2063] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:14 2022] DELETE /identity/v3/projects/aa9111175cad44b2bbc611b7ae36a991 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:14.494980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22230bc7-32b1-4a2d-b0ca-fe2cd28147ef None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.496022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22230bc7-32b1-4a2d-b0ca-fe2cd28147ef None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.496772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22230bc7-32b1-4a2d-b0ca-fe2cd28147ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.562505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-22230bc7-32b1-4a2d-b0ca-fe2cd28147ef None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:14.565922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-22230bc7-32b1-4a2d-b0ca-fe2cd28147ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:14.592444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:14.593225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.593769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.594448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.595650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:14.596373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:14.598634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:14.617535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1043/2064] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:14.624630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:14.628489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:14.656503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2da3a-f5a8-4f2f-8553-20e03141406c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:14.675458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:14.676982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.677505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.678007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.680337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:14.690302 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:14.717566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1022/2065] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:14.729782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.730438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.731373 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:14.766329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1044/2066] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:14 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:14.767837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:14.768712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:14.769422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:14.923565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1023/2067] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:14.925804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:14.970388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf3b81-1204-4ba5-becf-1995edf99c99 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:14.992126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:14.997070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:14.998028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:14.998557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.000107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:15.000678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.013501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125f691-0f99-437d-b075-d5958a476b51 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.016227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.030189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:15.054419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.061855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.062387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.062894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.067735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.081743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.091462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1024/2068] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:15.097624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.113676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:15.120980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d8fc-6540-41ca-b19c-613fdede5b22 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.145521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.146830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.147349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.147837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.149219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.153741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.155814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1045/2069] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:14 2022] POST /identity/v3/users => generated 288 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:15.162964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.186331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471b79d-e290-4c48-a920-0c4fd0fd4fec admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.195248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.196621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.197473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.198011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.199024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.203256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1025/2070] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:15 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:15.205116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.205590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.222859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1046/2071] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:15.234767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.284784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c09cd-8195-4377-8666-2a88d88b9a93 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.367416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffabc8-f9bf-44f4-be63-593ff423fc68 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.377170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.377901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.378191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.378477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.379210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.382702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.408510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.409376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.409692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.409967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.410754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.412648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.414019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1026/2072] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:15 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:15.420355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.444514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1047/2073] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:15.446450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.449456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34e6f1-b6ba-4c66-902a-7303cf9831ef admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.466055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e5d22-61c5-4510-85cd-f57c4fd3f8bc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.469756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.470836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.470836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.471139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.471947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.480393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e8823-e3b5-4bef-9292-2088ed680424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.482027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e8823-e3b5-4bef-9292-2088ed680424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.482516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e8823-e3b5-4bef-9292-2088ed680424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.482936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e8823-e3b5-4bef-9292-2088ed680424 admin admin] PATH_INFO: `/v3/projects/c08b1b7de3484b14b95155011491dc0b/users/04d35b8b19c946eeb4d870764cacc4c5/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.485210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.508434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e8823-e3b5-4bef-9292-2088ed680424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08b1b7de3484b14b95155011491dc0b, user_id=04d35b8b19c946eeb4d870764cacc4c5, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.516999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e8823-e3b5-4bef-9292-2088ed680424 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.519930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1027/2074] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:15.528557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.558439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1048/2075] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:15 2022] PUT /identity/v3/projects/c08b1b7de3484b14b95155011491dc0b/users/04d35b8b19c946eeb4d870764cacc4c5/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:15.564356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dad6e-a13f-4cfa-85f4-0d8ca52c74d0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.568622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.568929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.569216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.662125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] MFA Rules not processed for user `04d35b8b19c946eeb4d870764cacc4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:15.664413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:15.691884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.692942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.693771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.694398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.695119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.697762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.716642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1028/2076] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:15.717800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.730578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873881b-ff9a-49d4-97b6-9e3820509828 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.736298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af5cfd-ff65-444e-bd88-4a6bea63bbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:15.738276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5cfd-ff65-444e-bd88-4a6bea63bbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.738276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5cfd-ff65-444e-bd88-4a6bea63bbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.738579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5cfd-ff65-444e-bd88-4a6bea63bbdd admin admin] PATH_INFO: `/v3/projects/a4ce5005352f4759a1692401419579e5/users/4310a5779c084c83b6baa71af91109d7/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.779088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af5cfd-ff65-444e-bd88-4a6bea63bbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ce5005352f4759a1692401419579e5, user_id=4310a5779c084c83b6baa71af91109d7, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:15.781965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af5cfd-ff65-444e-bd88-4a6bea63bbdd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:15.831629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1029/2077] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:15 2022] PUT /identity/v3/projects/a4ce5005352f4759a1692401419579e5/users/4310a5779c084c83b6baa71af91109d7/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:15.844079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:15.844639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:15.844944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:15.858492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1049/2078] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:15.862938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:15.892276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b77463-f0d6-4ec9-bba0-20d38d7e55dc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:15.905874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] MFA Rules not processed for user `4310a5779c084c83b6baa71af91109d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:15.908588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:15.965776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:16.053719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:16.117865 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:16.118798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:16.119165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:16.119476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:16.129768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:16.131456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:16.220788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1030/2079] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:16.226309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:16.230908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1050/2080] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:15 2022] POST /identity/v3/projects => generated 379 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:16.252026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:16.290244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bcbec-3c12-420c-b4ef-14ef33d9c8cf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:16.528905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:16.531694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:16.532249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:16.532383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:16.533495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:16.542276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:16.554666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d54dae-bf14-438e-aabf-63fdd79a51ce admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:16.569983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:16.570720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:16.573237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:16.573237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:16.574407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:16.576537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:16.639274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1051/2081] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:16 2022] POST /identity/v3/users => generated 290 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:16.641565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:16.642794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1031/2082] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:16 2022] POST /identity/v3/projects => generated 379 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:16.647043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:16.667100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ced6e-4acf-4497-93cd-3bca358d7cfd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:16.862921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:16.862921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:16.863461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:16.863618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:16.864343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:16.878677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:16.907329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39f799-b66b-44f4-b5b2-d838110c0dcc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:16.907523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1052/2083] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:16.908691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:16.942602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46912-eb23-40f4-807c-344825d87ecc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:16.944313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:16.953748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:16.960587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:16.965190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:16.967930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:16.968469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:16.968924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:16.970024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:16.970438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:16.972316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:16.976607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:16.976860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.019873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1053/2084] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:17.024243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.031522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1032/2085] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:16 2022] POST /identity/v3/users => generated 296 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:17.036395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.042459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5441f-85a4-4f39-95dc-e5f42d734cfd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.057615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.058610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.058975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.059936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] PATH_INFO: `/v3/projects/2408397f65c740caa892b45e9f6ff19a/users/4bcd2a05d1d74dd7ab626bae1e48ba21/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.064546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861fc6b-642f-48fa-9100-5af3f40ad7c4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.086417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeede8-059c-43ea-9656-eb15cfd8573a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.089031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeede8-059c-43ea-9656-eb15cfd8573a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.089544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeede8-059c-43ea-9656-eb15cfd8573a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.090053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeede8-059c-43ea-9656-eb15cfd8573a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.091534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeede8-059c-43ea-9656-eb15cfd8573a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.096736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeede8-059c-43ea-9656-eb15cfd8573a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.106494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2408397f65c740caa892b45e9f6ff19a, user_id=4bcd2a05d1d74dd7ab626bae1e48ba21, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.109765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.131616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1054/2086] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:17 2022] PUT /identity/v3/projects/2408397f65c740caa892b45e9f6ff19a/users/4bcd2a05d1d74dd7ab626bae1e48ba21/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:17.136117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1033/2087] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:17.157750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.159791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.161366 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.163642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.225748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:17.290264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] MFA Rules not processed for user `4bcd2a05d1d74dd7ab626bae1e48ba21`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:17.295978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:17.312767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:17.376262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18572837-ec82-436f-8880-38ab3d1edbc8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.392549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.395185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.395502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.395784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] PATH_INFO: `/v3/projects/2be2876376eb4a64b3e7f0b0bb0a5460/users/5729e28434e142b4a67495d19d4edf9d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.456832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be2876376eb4a64b3e7f0b0bb0a5460, user_id=5729e28434e142b4a67495d19d4edf9d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.474076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.524810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1055/2088] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:17 2022] PUT /identity/v3/projects/2be2876376eb4a64b3e7f0b0bb0a5460/users/5729e28434e142b4a67495d19d4edf9d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:17.524912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1034/2089] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:17.534287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.534607 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.558393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea274c1-9a35-486d-a494-20e651a24da3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.678773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614b8e8-aee3-4a84-812e-fe630c093ba8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.688876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.689636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.690099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.690819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.691613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.694482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.716252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1056/2090] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:17.717298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.721608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.722514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.722815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.723071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.723976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.726512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.730403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a927c74-f6f6-4db8-8ff6-04ab19a30466 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.736957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17be2a8-02e9-4f81-bed6-9d037e5e0668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.737916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be2a8-02e9-4f81-bed6-9d037e5e0668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.738213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be2a8-02e9-4f81-bed6-9d037e5e0668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.738505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be2a8-02e9-4f81-bed6-9d037e5e0668 admin admin] PATH_INFO: `/v3/projects/2be2876376eb4a64b3e7f0b0bb0a5460/users/5729e28434e142b4a67495d19d4edf9d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.742337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17be2a8-02e9-4f81-bed6-9d037e5e0668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be2876376eb4a64b3e7f0b0bb0a5460, user_id=5729e28434e142b4a67495d19d4edf9d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.745777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17be2a8-02e9-4f81-bed6-9d037e5e0668 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.768084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1057/2091] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:17 2022] PUT /identity/v3/projects/2be2876376eb4a64b3e7f0b0bb0a5460/users/5729e28434e142b4a67495d19d4edf9d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:17.779605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.779918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.780293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.786367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1035/2092] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:17 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:17.792350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.820009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] MFA Rules not processed for user `5729e28434e142b4a67495d19d4edf9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:17.821516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:17.926002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e7dac-dba7-4afe-9bec-7f4867ec1940 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:17.931071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1058/2093] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:17.946664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:17.949461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:17.950615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:17.952250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:17.954857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:17.955312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:17.956899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:17.960240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:17.970958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:18.003552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1036/2094] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:17 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:18.008517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:18.031236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe685fff-8d64-4126-856e-17e38c60b5ba admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:18.042339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:18.044258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:18.044772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:18.045319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:18.048306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:18.053770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:18.127179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1037/2095] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:18.131423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:18.157188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359e8e-973c-48c9-935e-5cbd4e86431c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:18.174351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:18.175550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:18.176076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:18.176560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:18.183212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50cd16-dcf2-47cc-9a62-9d40d2092992 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:18.206694 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:18.208333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:18.208807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:18.209214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] PATH_INFO: `/v3/projects/017d8198cc2e4413a3da39840ddba3fa/users/c77228a6b44746e0a0692515b5202b47/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:18.219270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:18.220380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:18.235228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:18.263790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017d8198cc2e4413a3da39840ddba3fa, user_id=c77228a6b44746e0a0692515b5202b47, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:18.270887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:18.311652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1059/2096] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:18.341565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1038/2097] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:18 2022] PUT /identity/v3/projects/017d8198cc2e4413a3da39840ddba3fa/users/c77228a6b44746e0a0692515b5202b47/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:18.342806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:18.377746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd6ec4-4a76-43e7-881c-351851fe7362 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:18.575500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:18.576733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:18.577254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:18.577696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:18.579094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:18.584126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:18.621201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1060/2098] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:18.627329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:18.635919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:18.645183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170a0b9-5f0c-4da2-a05f-0b1819295f10 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:18.670972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a54b2-22bf-4e85-9776-5e5ff5babf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:18.672641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a54b2-22bf-4e85-9776-5e5ff5babf27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:18.674639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a54b2-22bf-4e85-9776-5e5ff5babf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:18.675308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a54b2-22bf-4e85-9776-5e5ff5babf27 admin admin] PATH_INFO: `/v3/projects/017d8198cc2e4413a3da39840ddba3fa/users/c77228a6b44746e0a0692515b5202b47/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:18.679608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a54b2-22bf-4e85-9776-5e5ff5babf27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017d8198cc2e4413a3da39840ddba3fa, user_id=c77228a6b44746e0a0692515b5202b47, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:18.685053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a54b2-22bf-4e85-9776-5e5ff5babf27 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:18.715727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1039/2099] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:18 2022] PUT /identity/v3/projects/017d8198cc2e4413a3da39840ddba3fa/users/c77228a6b44746e0a0692515b5202b47/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:18.736591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:18.740217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:18.742310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:18.797491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] MFA Rules not processed for user `c77228a6b44746e0a0692515b5202b47`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:18.799223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:18.891221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fea31-78eb-4536-85b3-52e6cfff70bf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:18.907887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1040/2100] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:18.953401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:18.957690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:19.032307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bf6ee-baa8-41af-948d-6c0b687742b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:19.034090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bf6ee-baa8-41af-948d-6c0b687742b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:19.034559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bf6ee-baa8-41af-948d-6c0b687742b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:19.034979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bf6ee-baa8-41af-948d-6c0b687742b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:19.067405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:19.108242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54584222-8ee5-4a95-9428-efa2a45bb7f7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:19.116230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:19.117426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:19.119784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:19.120450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:19.137324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:19.139530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:19.154827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1041/2101] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:19.162187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bf6ee-baa8-41af-948d-6c0b687742b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:19.165416 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bf6ee-baa8-41af-948d-6c0b687742b8 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:19.192754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1061/2102] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:19.223680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:19.238967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc590a-100a-4f54-bcb3-17431bf5e3dc service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:19.245581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:19.245581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:19.246071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:19.246263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:19.336780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:19.340811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:19.349097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:19.368324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1042/2103] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:22.720806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:22.721363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:22.721686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:22.779300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] MFA Rules not processed for user `0d7d57c4231b49b28a606a17e42267e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:22.780903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:22.911180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1062/2104] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:22.918671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:22.921097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:23.099837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db4ce7-906b-4005-a261-49234bc2d325 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:23.109183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:23.110159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:23.110449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:23.110729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:23.111456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:23.113671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:23.156096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1043/2105] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:22 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:23.158734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:23.177922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbd6de-0a94-4d9f-bf4e-2064def9be88 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:23.304451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:23.306898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:23.307558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:23.308489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:23.310673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:23.314983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:23.350145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1063/2106] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:23 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:23.354878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:23.368329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c536-3094-4893-8fdb-3969a77fcd58 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:23.374913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:23.375829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:23.376492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:23.376861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:23.377848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:23.380283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:23.398624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1044/2107] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:23.403903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:23.420776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204792c-ad9a-44bf-9a75-fda2b4db82fb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:23.427696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:23.429373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:23.429838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:23.430275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] PATH_INFO: `/v3/projects/36d2bd288b5c49b98df6e29aa13fa1f1/users/eae3c2dfa052478ebc0b63fafdebd141/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:23.453712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d2bd288b5c49b98df6e29aa13fa1f1, user_id=eae3c2dfa052478ebc0b63fafdebd141, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:23.456625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:23.477181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1064/2108] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:23 2022] PUT /identity/v3/projects/36d2bd288b5c49b98df6e29aa13fa1f1/users/eae3c2dfa052478ebc0b63fafdebd141/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:23.481964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:23.492650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f591-05d7-47f6-9eb4-d396659a7228 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:23.575743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:23.576877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:23.577347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:23.577812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:23.579281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:23.582479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:23.602814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1045/2109] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:23.608918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:23.629390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075864a8-570b-4372-9298-e3f90cace294 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:23.640240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342a56a-47e9-4439-b66b-4d76343832f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:23.641838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342a56a-47e9-4439-b66b-4d76343832f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:23.642257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342a56a-47e9-4439-b66b-4d76343832f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:23.644592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342a56a-47e9-4439-b66b-4d76343832f1 admin admin] PATH_INFO: `/v3/projects/36d2bd288b5c49b98df6e29aa13fa1f1/users/eae3c2dfa052478ebc0b63fafdebd141/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:23.646805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342a56a-47e9-4439-b66b-4d76343832f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d2bd288b5c49b98df6e29aa13fa1f1, user_id=eae3c2dfa052478ebc0b63fafdebd141, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:23.653851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342a56a-47e9-4439-b66b-4d76343832f1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:23.670171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:23.678222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1065/2110] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:23 2022] PUT /identity/v3/projects/36d2bd288b5c49b98df6e29aa13fa1f1/users/eae3c2dfa052478ebc0b63fafdebd141/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:26.049737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:26.069423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34695a-2c26-482c-8069-ab4886eedca0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:26.256872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:26.261024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:26.261657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:26.262707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] PATH_INFO: `/v3/users/04d35b8b19c946eeb4d870764cacc4c5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:26.265624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d35b8b19c946eeb4d870764cacc4c5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:26.271657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:26.307583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:26.474219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d35b8b19c946eeb4d870764cacc4c5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:26.511379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d35b8b19c946eeb4d870764cacc4c5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:26.557854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d35b8b19c946eeb4d870764cacc4c5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:26.608636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d35b8b19c946eeb4d870764cacc4c5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:26.609750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d35b8b19c946eeb4d870764cacc4c5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:26.630534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1046/2111] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:26 2022] DELETE /identity/v3/users/04d35b8b19c946eeb4d870764cacc4c5 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:26.632932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:26.634010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:26.635031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:26.700970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:26.706962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:26.857957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1066/2112] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:26.928769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:26.931729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:27.166573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98817b29-d476-42e0-9f8a-e7e28617a73d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:27.171597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:27.172377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:27.172748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:27.173118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:27.193019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:27.201208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:27.242040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1047/2113] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:28.731905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:28.735498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:28.761503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ac8a3-ef5f-49b6-ad3f-f038b30f34d0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:28.768765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:28.777672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:28.779099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:28.779602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] PATH_INFO: `/v3/projects/c08b1b7de3484b14b95155011491dc0b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:28.783679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08b1b7de3484b14b95155011491dc0b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:28.788319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:28.927881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08b1b7de3484b14b95155011491dc0b', 'request_id': 'req-f4ef99de-41ff-4001-8cf4-9b104639f564'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:28.938771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08b1b7de3484b14b95155011491dc0b', 'request_id': 'req-f4ef99de-41ff-4001-8cf4-9b104639f564'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:28.972142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08b1b7de3484b14b95155011491dc0b', 'request_id': 'req-f4ef99de-41ff-4001-8cf4-9b104639f564'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:28.994128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] The token cache is being invalidate because project c08b1b7de3484b14b95155011491dc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:28.995077 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:28.996564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:28.999330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1067/2114] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:28 2022] DELETE /identity/v3/projects/c08b1b7de3484b14b95155011491dc0b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:29.010547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:29.040662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6beb-640c-4763-9cf3-f4527d15ee13 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:29.222773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:29.224148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:29.224603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:29.224957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] PATH_INFO: `/v3/users/4bcd2a05d1d74dd7ab626bae1e48ba21` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:29.226474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bcd2a05d1d74dd7ab626bae1e48ba21)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:29.230486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:29.347511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:29.401118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bcd2a05d1d74dd7ab626bae1e48ba21'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:29.432497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bcd2a05d1d74dd7ab626bae1e48ba21'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:29.473132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bcd2a05d1d74dd7ab626bae1e48ba21'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:29.536886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bcd2a05d1d74dd7ab626bae1e48ba21'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:29.537878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bcd2a05d1d74dd7ab626bae1e48ba21'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:29.560107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1048/2115] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:29 2022] DELETE /identity/v3/users/4bcd2a05d1d74dd7ab626bae1e48ba21 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:30.447943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:30.450264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:30.505062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef99de-41ff-4001-8cf4-9b104639f564 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:30.758786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:30.761189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:30.761189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:30.761528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] PATH_INFO: `/v3/projects/2408397f65c740caa892b45e9f6ff19a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:30.763235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2408397f65c740caa892b45e9f6ff19a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:30.765767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:30.874994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:30.904354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2408397f65c740caa892b45e9f6ff19a', 'request_id': 'req-ff5273b3-1d20-4761-8c79-b43246b79cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:30.912790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2408397f65c740caa892b45e9f6ff19a', 'request_id': 'req-ff5273b3-1d20-4761-8c79-b43246b79cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:30.922410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2408397f65c740caa892b45e9f6ff19a', 'request_id': 'req-ff5273b3-1d20-4761-8c79-b43246b79cee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:30.955481 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] The token cache is being invalidate because project 2408397f65c740caa892b45e9f6ff19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:30.956275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:30.957454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:30.961852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1068/2116] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:30 2022] DELETE /identity/v3/projects/2408397f65c740caa892b45e9f6ff19a => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:30.968265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:30.970762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:30.990882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b24b1-8dad-41b8-83a8-59b1c4a2f65a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:31.122550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:31.127214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:31.128545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:31.129165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] PATH_INFO: `/v3/users/c77228a6b44746e0a0692515b5202b47` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:31.130936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77228a6b44746e0a0692515b5202b47)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:31.136209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:31.276130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77228a6b44746e0a0692515b5202b47'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:31.302676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77228a6b44746e0a0692515b5202b47'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:31.326002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77228a6b44746e0a0692515b5202b47'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:31.348740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77228a6b44746e0a0692515b5202b47'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:31.349663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5f699837-7636-4c9c-900f-57876ad86cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77228a6b44746e0a0692515b5202b47'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:31.361969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1049/2117] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:30 2022] DELETE /identity/v3/users/c77228a6b44746e0a0692515b5202b47 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:32.245156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:32.247683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:32.319260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5273b3-1d20-4761-8c79-b43246b79cee admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:32.447113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:32.449075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:32.449730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:32.450417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] PATH_INFO: `/v3/projects/017d8198cc2e4413a3da39840ddba3fa` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:32.453066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=017d8198cc2e4413a3da39840ddba3fa)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:32.456916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:32.586069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017d8198cc2e4413a3da39840ddba3fa', 'request_id': 'req-0b79509e-73f9-4406-b59b-15bdaf6802cc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:32.601909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017d8198cc2e4413a3da39840ddba3fa', 'request_id': 'req-0b79509e-73f9-4406-b59b-15bdaf6802cc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:32.616030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017d8198cc2e4413a3da39840ddba3fa', 'request_id': 'req-0b79509e-73f9-4406-b59b-15bdaf6802cc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:32.653476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] The token cache is being invalidate because project 017d8198cc2e4413a3da39840ddba3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:32.654734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:32.665465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0b79509e-73f9-4406-b59b-15bdaf6802cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:32.669779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1069/2118] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:32 2022] DELETE /identity/v3/projects/017d8198cc2e4413a3da39840ddba3fa => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:32.679970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:32.680650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:32.682260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:32.728790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:32.731616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:32.817916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:32.948787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1050/2119] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:32.961891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:32.963060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:32.963060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:32.998284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:32.999901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:33.022853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.032039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1070/2120] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:33.047405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d25dd-7de0-412c-aa53-d12310eb0b2e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.070146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.075273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.076497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.077684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.080245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:33.081322 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:33.088653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:33.125253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1051/2121] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:33.130267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.156285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e297b89-cf92-4375-a464-69b7094acaae None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.179642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895485f-7bd2-4456-be68-2207a1c71269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.180834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895485f-7bd2-4456-be68-2207a1c71269 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.181521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895485f-7bd2-4456-be68-2207a1c71269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.181911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895485f-7bd2-4456-be68-2207a1c71269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.183287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895485f-7bd2-4456-be68-2207a1c71269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:33.186319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895485f-7bd2-4456-be68-2207a1c71269 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:33.270961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1071/2122] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:33 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:33.272634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.292521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.294129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.294129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.303301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8297115-5e7b-454c-a318-172721fb351e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.345843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] MFA Rules not processed for user `eae3c2dfa052478ebc0b63fafdebd141`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:33.347472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:33.354505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:33.507964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.509888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.512516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.513086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.514401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:33.538236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:33.590721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1052/2123] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:33 2022] POST /identity/v3/users => generated 283 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:33.597647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.625679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1072/2124] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:33.634653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66833c-2922-49aa-bdb7-c0ecb9a893e5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.657148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.663493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:33.680807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.683015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.683776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.684779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.687378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:33.692726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:33.747871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1053/2125] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:33.758347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.849169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ededb-abb8-44e8-9584-a667517c80c7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.865402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.866338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.866775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.867209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.868757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:33.870889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:33.908874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1054/2126] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:33.913909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:33.933512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf9b85-11b3-44ba-8c67-9e4dbd760d24 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.948854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30455dc6-67be-4550-97c2-9e015e4d2e7b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:33.959272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.963703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f6dad-7dfd-4be0-9a41-ca8f139a0602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:33.965451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6dad-7dfd-4be0-9a41-ca8f139a0602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.965861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:33.965996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6dad-7dfd-4be0-9a41-ca8f139a0602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.966388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:33.966543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6dad-7dfd-4be0-9a41-ca8f139a0602 admin admin] PATH_INFO: `/v3/projects/3c6059c3c7e445af93e821f0275cf727/users/7effb9831ccc4655a5c49a2315db9b81/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.967045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:33.971589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:34.010286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f6dad-7dfd-4be0-9a41-ca8f139a0602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6059c3c7e445af93e821f0275cf727, user_id=7effb9831ccc4655a5c49a2315db9b81, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:34.015260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f6dad-7dfd-4be0-9a41-ca8f139a0602 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:34.045090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1055/2127] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:33 2022] PUT /identity/v3/projects/3c6059c3c7e445af93e821f0275cf727/users/7effb9831ccc4655a5c49a2315db9b81/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:34.062181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ac483-7c78-4eaa-aa8e-2ef520684d12 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:34.062808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ac483-7c78-4eaa-aa8e-2ef520684d12 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:34.063605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ac483-7c78-4eaa-aa8e-2ef520684d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:34.132137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a69ac483-7c78-4eaa-aa8e-2ef520684d12 None None] MFA Rules not processed for user `7effb9831ccc4655a5c49a2315db9b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:34.134595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a69ac483-7c78-4eaa-aa8e-2ef520684d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:34.170001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:34.176465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:34.213054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1073/2128] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:34.298973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:34.360118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1056/2129] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 18:30:34.367817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:34.389256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c7af3-e0d1-46ed-8d7a-3d3e89e0fdd8 service glance] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:34.433292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:34.433806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:34.434347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:34.462339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:34.465611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:34.512974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:34.527236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:34.529613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:34.530005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:34.531244 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:34.532559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:34.535428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:34.608680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1074/2130] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:34 2022] POST /identity/v3/projects => generated 367 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:34.612407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1057/2131] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:30:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:34.615986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:34.628564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:34.630723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:34.879094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334d429-0dd0-4cd6-bbc6-bc41e3a68ca6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:34.898511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:34.899649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:34.900279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:34.900947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:34.901877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:34.906923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:34.921253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a60bc0-38c3-4b65-8c1a-d0697a7fedf9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:34.984159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1075/2132] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:34 2022] POST /identity/v3/users => generated 290 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:34.989351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:35.038226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5444b-5655-4de0-8465-7a17200bcbb7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:35.053805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:35.055148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:35.055510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:35.056157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:35.057354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:35.062217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:35.093494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1076/2133] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:35.110166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:35.139905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56b9a2-b64f-4209-91b4-ce30896d6dba admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:35.149069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:35.150870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:35.152023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:35.152572 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:35.189944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:35.191382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:35.191812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:35.192309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] PATH_INFO: `/v3/projects/f9f11b8e82cf4c19b45462748694e496/users/f6758eaf837a47b8b6e791dad519e276/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:35.276687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f11b8e82cf4c19b45462748694e496, user_id=f6758eaf837a47b8b6e791dad519e276, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:35.286318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:35.320306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:35.412163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:35.415860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:35.436131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1077/2134] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:35 2022] PUT /identity/v3/projects/f9f11b8e82cf4c19b45462748694e496/users/f6758eaf837a47b8b6e791dad519e276/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:35.447055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:35.457215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1058/2135] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:30:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 830 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:30:35.828333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:35.885943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1584c7-508a-466e-81f3-3bc5467b0adf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:35.903685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37b592-b94f-4aff-9bcd-2048610e11ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:35.906354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37b592-b94f-4aff-9bcd-2048610e11ff admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:35.906521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37b592-b94f-4aff-9bcd-2048610e11ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:35.907289 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37b592-b94f-4aff-9bcd-2048610e11ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:35.909525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37b592-b94f-4aff-9bcd-2048610e11ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:35.914796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37b592-b94f-4aff-9bcd-2048610e11ff admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:35.965013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1078/2136] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:35.967949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:36.141067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad3011-e329-4398-b76e-b2f3218607f5 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:36.163894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:36.165594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:36.166955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:36.167450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] PATH_INFO: `/v3/projects/f9f11b8e82cf4c19b45462748694e496/users/f6758eaf837a47b8b6e791dad519e276/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:36.195128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f11b8e82cf4c19b45462748694e496, user_id=f6758eaf837a47b8b6e791dad519e276, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:36.205983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:36.272871 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1059/2137] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:35 2022] PUT /identity/v3/projects/f9f11b8e82cf4c19b45462748694e496/users/f6758eaf837a47b8b6e791dad519e276/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:36.291656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:36.291952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:36.292308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:36.407037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] MFA Rules not processed for user `f6758eaf837a47b8b6e791dad519e276`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:36.408372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:36.733657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1079/2138] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:37.136623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:37.141609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:37.411747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512752a-ec61-495a-aea4-76a324a563cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:37.420534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:37.421705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:37.421925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:37.422186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:37.462175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:37.467828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:37.500217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1060/2139] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:38.191981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:38.228962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09353096-47ba-40c5-ae7e-59b781c293bf None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:38.248149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a11c3-3d26-4ad1-9e2e-b266742208dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:38.249303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a11c3-3d26-4ad1-9e2e-b266742208dd service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:38.249776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a11c3-3d26-4ad1-9e2e-b266742208dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:38.250253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a11c3-3d26-4ad1-9e2e-b266742208dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:38.538704 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:38.543726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:38.559397 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a11c3-3d26-4ad1-9e2e-b266742208dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:38.577079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a11c3-3d26-4ad1-9e2e-b266742208dd service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:38.639853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1080/2140] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:38.941050 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1e208-256b-4082-b5e6-86220425c000 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:39.014345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:39.025914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:39.025914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:39.026425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] PATH_INFO: `/v3/users/3287fc466d684c3a8444493f2ff12aee` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:39.064593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3287fc466d684c3a8444493f2ff12aee)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:39.096696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:39.271566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:39.435494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3287fc466d684c3a8444493f2ff12aee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:39.467702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3287fc466d684c3a8444493f2ff12aee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:39.527651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3287fc466d684c3a8444493f2ff12aee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:39.625952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3287fc466d684c3a8444493f2ff12aee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:39.635552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3287fc466d684c3a8444493f2ff12aee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:39.664410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1061/2141] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:38 2022] DELETE /identity/v3/users/3287fc466d684c3a8444493f2ff12aee => generated 0 bytes in 1127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:39.672941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:39.673680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:39.673680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:39.824290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:39.840988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:40.021174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:40.026624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:40.132444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:40.237520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1081/2142] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:40.254663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:40.258023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:40.290359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad397bc0-e1ca-4151-a8fd-ec6b4a82cb48 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:40.300218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:40.301154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:40.302276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:40.302538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] PATH_INFO: `/v3/users/4310a5779c084c83b6baa71af91109d7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:40.303936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4310a5779c084c83b6baa71af91109d7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:40.306671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:40.538683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4310a5779c084c83b6baa71af91109d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:40.558775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4310a5779c084c83b6baa71af91109d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:40.580969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4310a5779c084c83b6baa71af91109d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:40.623921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4310a5779c084c83b6baa71af91109d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:40.625069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3bcbc0a5-9825-45a4-8f8a-f605c74318a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4310a5779c084c83b6baa71af91109d7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:40.645587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1062/2143] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:40 2022] DELETE /identity/v3/users/4310a5779c084c83b6baa71af91109d7 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:40.647163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:40.647456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:40.647736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:40.705330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74836285-14bf-43f2-8ad5-7a0ba1d03cfa None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:40.726995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:40.728664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:40.806537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:40.876167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:40.877158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:40.879075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:40.879719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:40.891247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:40.918021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1063/2144] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:40.919137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:40.920843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:40.945075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1082/2145] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:40.998264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:40.999924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:41.045443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7cf84-2a06-4cb7-b0e3-12eb7ff01681 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:41.071776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:41.074456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:41.074818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:41.088740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:41.175348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:41.175348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:41.282550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1064/2146] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:43.705693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:43.708007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:43.744135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb50c2e-f7dc-4f9e-892c-4ccd8e10c35c service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:43.769705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:43.782315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:43.783186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:43.787911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] PATH_INFO: `/v3/projects/6d91db54b9c2472eb44f90c7d6f4c411` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:43.803185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d91db54b9c2472eb44f90c7d6f4c411)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:43.811062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:43.954218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:43.958251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:44.050600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b8449-9fc6-4638-a335-cd63466ace68 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:44.112807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:44.114730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:44.115577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:44.116116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] PATH_INFO: `/v3/projects/a4ce5005352f4759a1692401419579e5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:44.121328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ce5005352f4759a1692401419579e5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:44.126118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:44.276492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d91db54b9c2472eb44f90c7d6f4c411', 'request_id': 'req-6741309c-696c-4a33-bcc9-941ac43bd50c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.324584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d91db54b9c2472eb44f90c7d6f4c411', 'request_id': 'req-6741309c-696c-4a33-bcc9-941ac43bd50c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.361166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d91db54b9c2472eb44f90c7d6f4c411', 'request_id': 'req-6741309c-696c-4a33-bcc9-941ac43bd50c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.408148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ce5005352f4759a1692401419579e5', 'request_id': 'req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.416633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] The token cache is being invalidate because project 6d91db54b9c2472eb44f90c7d6f4c411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:44.418966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.420524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6741309c-696c-4a33-bcc9-941ac43bd50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.436819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1083/2147] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:43 2022] DELETE /identity/v3/projects/6d91db54b9c2472eb44f90c7d6f4c411 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:44.438448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60572-f937-4850-833a-ca70e7419d0b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:44.438738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60572-f937-4850-833a-ca70e7419d0b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:44.439016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60572-f937-4850-833a-ca70e7419d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:44.442200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ce5005352f4759a1692401419579e5', 'request_id': 'req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.470522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ce5005352f4759a1692401419579e5', 'request_id': 'req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.531844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] The token cache is being invalidate because project a4ce5005352f4759a1692401419579e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:44.535893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.538456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:44.548112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:44.550696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1065/2148] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:43 2022] DELETE /identity/v3/projects/a4ce5005352f4759a1692401419579e5 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:44.555078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:44.562674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:44.737324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7ad60572-f937-4850-833a-ca70e7419d0b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:44.745193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7ad60572-f937-4850-833a-ca70e7419d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:44.960424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0a061-6e7c-4d70-bfa8-48d6e23783f6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:44.997616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:44.998941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:44.999678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:45.001431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] PATH_INFO: `/v3/users/5729e28434e142b4a67495d19d4edf9d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:45.004024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5729e28434e142b4a67495d19d4edf9d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:45.023025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:45.052139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1084/2149] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 610 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:45.054700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:45.072660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:45.073668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:45.169856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:45.171634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:45.239541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1085/2150] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:45.243084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:45.268979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:45.304646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5729e28434e142b4a67495d19d4edf9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:45.312769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5729e28434e142b4a67495d19d4edf9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:45.329014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccaebe-76ef-4a0a-bc5a-f99682a45f04 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:45.345320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:45.346154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:45.346546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:45.347597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:45.349651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:45.350410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5729e28434e142b4a67495d19d4edf9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:45.351352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:45.362141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:45.436390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5729e28434e142b4a67495d19d4edf9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:45.437912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5729e28434e142b4a67495d19d4edf9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:45.462770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1066/2151] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:44 2022] DELETE /identity/v3/users/5729e28434e142b4a67495d19d4edf9d => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:45.519401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1086/2152] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:45.524121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:45.527292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:45.889830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd70a8-467d-463d-9545-84cf5e4828ac admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:45.920104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:45.921488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:45.922064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:45.928939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:45.936762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:45.939857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:46.039930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1067/2153] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:45 2022] POST /identity/v3/projects => generated 373 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:46.044040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:46.081407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63fde9-5b48-43ba-8864-9da3a6cf4aec admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:46.393516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:46.394844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:46.395798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:46.398618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:46.409126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:46.418323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:46.603698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1087/2154] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:46 2022] POST /identity/v3/users => generated 287 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:46.606515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:46.695203 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f02c-f09d-4db4-a70b-59f06c3dfa04 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:46.722525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:46.728568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:46.728732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:46.729224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:46.730757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:46.738608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:46.790415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:46.831616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1068/2155] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:46.842821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:46.844985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:46.882560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e375-0e02-4be7-abf5-920e79c3b12f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:46.892935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:46.894197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:46.894707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:46.895188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:46.897074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:46.905511 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:46.955447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64f75-3733-480e-93ee-b38d98863f7b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:46.976119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1069/2156] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:46.985218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:47.032925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:47.034528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:47.034985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:47.035491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] PATH_INFO: `/v3/projects/2be2876376eb4a64b3e7f0b0bb0a5460` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:47.050882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be2876376eb4a64b3e7f0b0bb0a5460)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:47.052779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d36e30-e05e-498f-87cd-dace1725258a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:47.058400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:47.086224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d9c22-ae21-47a3-a902-c5d94c276d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:47.089520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d9c22-ae21-47a3-a902-c5d94c276d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:47.093621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d9c22-ae21-47a3-a902-c5d94c276d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:47.093621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d9c22-ae21-47a3-a902-c5d94c276d62 admin admin] PATH_INFO: `/v3/projects/e153f3946d3e414fb5931013f9f385c5/users/3c00ab18c6db4a70b972c320319585d2/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:47.159048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d9c22-ae21-47a3-a902-c5d94c276d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e153f3946d3e414fb5931013f9f385c5, user_id=3c00ab18c6db4a70b972c320319585d2, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:47.172848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d9c22-ae21-47a3-a902-c5d94c276d62 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:47.238915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1070/2157] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:46 2022] PUT /identity/v3/projects/e153f3946d3e414fb5931013f9f385c5/users/3c00ab18c6db4a70b972c320319585d2/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:47.251260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:47.251555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:47.251817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:47.257934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be2876376eb4a64b3e7f0b0bb0a5460', 'request_id': 'req-fb6ee85a-dc53-478a-9dc9-33055b8d9867'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:47.289934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be2876376eb4a64b3e7f0b0bb0a5460', 'request_id': 'req-fb6ee85a-dc53-478a-9dc9-33055b8d9867'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:47.366195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:47.367379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] MFA Rules not processed for user `3c00ab18c6db4a70b972c320319585d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:47.369747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:47.387083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be2876376eb4a64b3e7f0b0bb0a5460', 'request_id': 'req-fb6ee85a-dc53-478a-9dc9-33055b8d9867'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:47.478676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] The token cache is being invalidate because project 2be2876376eb4a64b3e7f0b0bb0a5460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:47.479368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:47.482011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb6ee85a-dc53-478a-9dc9-33055b8d9867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:47.489293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1088/2158] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:46 2022] DELETE /identity/v3/projects/2be2876376eb4a64b3e7f0b0bb0a5460 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:47.498848 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde647b5-ab72-489d-b1a6-3eb4521c3958 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:47.499320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde647b5-ab72-489d-b1a6-3eb4521c3958 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:47.512230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde647b5-ab72-489d-b1a6-3eb4521c3958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:47.555024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:47.631982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fde647b5-ab72-489d-b1a6-3eb4521c3958 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:47.639010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fde647b5-ab72-489d-b1a6-3eb4521c3958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:47.661887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1071/2159] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:47.693782 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:47.697201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:47.885324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e981e-fabe-454c-a58a-c874d6022eab None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:47.894016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:47.895903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:47.897793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:47.898459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:47.911729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1089/2160] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:47.914419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:47.914952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:47.916297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:47.932658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:47.941992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.016490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:48.026676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:48.044007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1090/2161] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:48.045138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1072/2162] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:48.049490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:48.070831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92ebe9-cf80-41a9-91d4-9ca54a15ee84 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:48.082154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:48.083320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:48.083897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:48.084556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:48.085643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:48.086257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:48.095715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.174036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1091/2163] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:48.180017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:48.198751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c74be-56a5-4df8-adf3-5c5bc27801bb service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:48.207450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:48.208184 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:48.208472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:48.208746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:48.209489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:48.213551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.278252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1073/2164] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:48 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:48.284277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:48.303343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887ce2-9ab3-4e21-97e6-f21d18dff228 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:48.451614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:48.451614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:48.451614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:48.451614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:48.453041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:48.455819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.526562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1092/2165] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:48 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:48.534718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:48.555170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5a27e-e23b-4d47-8830-f9f13058cbaa admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:48.573058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:48.575877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:48.576270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:48.576550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:48.577333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:48.579779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.664195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1074/2166] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:48.673080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:48.710727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b434f25-5632-44b3-af31-72d13374de4f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:48.740752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163b921-aa1c-41cc-bbb0-8bbaf0566c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:48.741521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163b921-aa1c-41cc-bbb0-8bbaf0566c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:48.741809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163b921-aa1c-41cc-bbb0-8bbaf0566c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:48.742089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163b921-aa1c-41cc-bbb0-8bbaf0566c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:48.743016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163b921-aa1c-41cc-bbb0-8bbaf0566c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:48.751144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163b921-aa1c-41cc-bbb0-8bbaf0566c7e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.799968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1093/2167] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:48.805596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:48.834113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826cd14-12a0-49f3-b6a0-28fa1b161ebb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:48.864161 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:48.865120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:48.873480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:48.878988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] PATH_INFO: `/v3/projects/366f74fa3db640e3854cbdeef105ec8c/users/46ad25da636b46fa8fbd8ea00deb2693/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:48.933074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] RBAC: Authorizing `identity:create_grant(project_id=366f74fa3db640e3854cbdeef105ec8c, user_id=46ad25da636b46fa8fbd8ea00deb2693, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:48.939222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:48.989530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1075/2168] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:48 2022] PUT /identity/v3/projects/366f74fa3db640e3854cbdeef105ec8c/users/46ad25da636b46fa8fbd8ea00deb2693/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:30:49.003439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:49.003740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:49.004189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:49.083070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] MFA Rules not processed for user `46ad25da636b46fa8fbd8ea00deb2693`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:49.084367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:49.293921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:49.456551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1094/2169] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:49.463943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:49.536278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:49.544018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:49.661866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b619009-a8e6-457c-95be-cebf178733af admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:49.689408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:49.690225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:49.690514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:49.690879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:49.691830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:49.695734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:49.781014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1076/2170] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:49 2022] POST /identity/v3/projects => generated 369 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:49.786432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:49.842345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:49.903367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e08bdd-51d3-4282-ac00-9e201cdf271c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:49.990397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d85c8a-4da0-4bc4-befd-07cf1689ef7b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:50.005910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:50.007730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:50.011838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:50.012434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:50.013376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:50.017368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:50.054251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1077/2171] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:49 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:50.061728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:50.062660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:50.063209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:50.063209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] PATH_INFO: `/v3/users/7effb9831ccc4655a5c49a2315db9b81` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:50.065435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7effb9831ccc4655a5c49a2315db9b81)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:50.067484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:50.070944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:50.096269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e0ea0-4706-49da-8508-fe51074f89c7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:50.119011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb5dc9-6b07-486b-94f2-47b76cf7b13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:50.121299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5dc9-6b07-486b-94f2-47b76cf7b13f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:50.121950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5dc9-6b07-486b-94f2-47b76cf7b13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:50.122551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5dc9-6b07-486b-94f2-47b76cf7b13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:50.123956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb5dc9-6b07-486b-94f2-47b76cf7b13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:50.127663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb5dc9-6b07-486b-94f2-47b76cf7b13f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:50.155774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1078/2172] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:50.158921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:50.159689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:50.161400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:50.201213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7effb9831ccc4655a5c49a2315db9b81'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:50.218635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] MFA Rules not processed for user `89f6c755dbf74640a372802f02733eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:50.225183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:50.253059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7effb9831ccc4655a5c49a2315db9b81'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:50.289477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7effb9831ccc4655a5c49a2315db9b81'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:50.293142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7effb9831ccc4655a5c49a2315db9b81'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:50.295354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b53143e3-60a2-4bb8-b9ad-808e517e7d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7effb9831ccc4655a5c49a2315db9b81'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:50.337345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1079/2173] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:50.342624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:50.343917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1095/2174] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:49 2022] DELETE /identity/v3/users/7effb9831ccc4655a5c49a2315db9b81 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:50.346859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:50.350105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:50.350525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:50.350925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:50.376374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:50.398777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96074a-ab43-4c21-90df-2daf4adb954c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:50.421203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:50.422452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:50.601919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:50.603658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:50.604706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:50.605421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] PATH_INFO: `/v3/projects/297adc38bf7f447e85d6fccfac29817b/users/0fbf4c2c4d88422e9473a02df4058393/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:50.635817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=297adc38bf7f447e85d6fccfac29817b, user_id=0fbf4c2c4d88422e9473a02df4058393, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:50.645677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:50.699174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1096/2175] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:50.717568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:50.719722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:50.741125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1080/2176] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:50 2022] PUT /identity/v3/projects/297adc38bf7f447e85d6fccfac29817b/users/0fbf4c2c4d88422e9473a02df4058393/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:50.747200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:50.831933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:50.984672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb69d0-324c-4b0a-9432-055be65b2c1e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:51.017157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:51.019068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:51.019497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:51.030824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:51.032593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:51.035759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:51.040749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55cf3b-e895-4514-909f-370a325e7833 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:51.116410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1081/2177] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:51.118374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:51.151638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687fc2c-8e62-4cf4-aa16-d6290b783471 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:51.161231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28c246-4f17-4ea4-a7a4-d297b7a6eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:51.162282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c246-4f17-4ea4-a7a4-d297b7a6eccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:51.162675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c246-4f17-4ea4-a7a4-d297b7a6eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:51.163021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c246-4f17-4ea4-a7a4-d297b7a6eccf admin admin] PATH_INFO: `/v3/projects/297adc38bf7f447e85d6fccfac29817b/users/0fbf4c2c4d88422e9473a02df4058393/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:51.168477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c246-4f17-4ea4-a7a4-d297b7a6eccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=297adc38bf7f447e85d6fccfac29817b, user_id=0fbf4c2c4d88422e9473a02df4058393, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:51.175255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c246-4f17-4ea4-a7a4-d297b7a6eccf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:51.226301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1082/2178] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:51 2022] PUT /identity/v3/projects/297adc38bf7f447e85d6fccfac29817b/users/0fbf4c2c4d88422e9473a02df4058393/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:51.250261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:51.251340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:51.253187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:51.253980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:51.258805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:51.258805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:51.260924 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:51.308590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:51.316938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:51.351851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] MFA Rules not processed for user `0fbf4c2c4d88422e9473a02df4058393`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:51.356553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1097/2179] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:30:51.369991 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:51.862106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1083/2180] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 606 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:51.887732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:51.890593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:52.146621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9e965-b2ab-4aec-9331-2ad01f280e80 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:52.200762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:52.387205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:52.388767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:52.389287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:52.389933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:52.438720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:52.443858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:52.475347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1098/2181] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:52.989296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:52.993900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:53.409634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516d4ad-8e5b-4df5-8c15-adceb52aab2c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:53.468726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:53.470341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:53.470771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:53.471324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] PATH_INFO: `/v3/projects/3c6059c3c7e445af93e821f0275cf727` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:53.499638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6059c3c7e445af93e821f0275cf727)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:53.505542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:53.656861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:53.768949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6059c3c7e445af93e821f0275cf727', 'request_id': 'req-019d26a3-78a1-4b47-82ad-b4a42f50179a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:53.804505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6059c3c7e445af93e821f0275cf727', 'request_id': 'req-019d26a3-78a1-4b47-82ad-b4a42f50179a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:53.840139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6059c3c7e445af93e821f0275cf727', 'request_id': 'req-019d26a3-78a1-4b47-82ad-b4a42f50179a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:53.905102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] The token cache is being invalidate because project 3c6059c3c7e445af93e821f0275cf727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:53.916175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:53.918060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:53.932084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1084/2182] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:52 2022] DELETE /identity/v3/projects/3c6059c3c7e445af93e821f0275cf727 => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:53.938527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:54.003975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50938dd-4404-4d8d-ab95-1819e72e14bf service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:54.210710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:54.214614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:54.215769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:54.216835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] PATH_INFO: `/v3/users/f6758eaf837a47b8b6e791dad519e276` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:54.220567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6758eaf837a47b8b6e791dad519e276)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:54.226560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:54.436339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6758eaf837a47b8b6e791dad519e276'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:54.474075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6758eaf837a47b8b6e791dad519e276'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:54.505090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6758eaf837a47b8b6e791dad519e276'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:54.507748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6758eaf837a47b8b6e791dad519e276'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:54.514813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-deb1fd5a-dbdb-4716-b736-bf256a11c260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6758eaf837a47b8b6e791dad519e276'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:54.568519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1099/2183] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:53 2022] DELETE /identity/v3/users/f6758eaf837a47b8b6e791dad519e276 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:55.567682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:55.570067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:55.600425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d26a3-78a1-4b47-82ad-b4a42f50179a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:55.961439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:55.963349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:55.971028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:55.971539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] PATH_INFO: `/v3/projects/f9f11b8e82cf4c19b45462748694e496` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:55.975705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f11b8e82cf4c19b45462748694e496)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:55.980605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:56.333596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f11b8e82cf4c19b45462748694e496', 'request_id': 'req-0f1244af-4db1-4541-a199-c7002be1d2cf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:56.398075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f11b8e82cf4c19b45462748694e496', 'request_id': 'req-0f1244af-4db1-4541-a199-c7002be1d2cf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:56.433274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f11b8e82cf4c19b45462748694e496', 'request_id': 'req-0f1244af-4db1-4541-a199-c7002be1d2cf'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:56.500208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] The token cache is being invalidate because project f9f11b8e82cf4c19b45462748694e496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:30:56.501009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:56.503440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0f1244af-4db1-4541-a199-c7002be1d2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:56.512713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1085/2184] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:30:55 2022] DELETE /identity/v3/projects/f9f11b8e82cf4c19b45462748694e496 => generated 0 bytes in 946 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:56.528565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:56.530454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:56.532102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:56.683019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:56.689166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:56.841697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:56.900962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1100/2185] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:56.908605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:56.909528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:56.909971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:56.959484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:56.962284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:56.973867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1086/2186] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:30:56.978829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:56.995714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a42196-6bed-4a83-a275-b0947f78e0d8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:57.002687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:57.003551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.004243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.004639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:57.005723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:30:57.006144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:57.007956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:57.027426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1101/2187] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:30:57.032913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:57.056723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd581f-0d13-4bb9-9d5d-2ac8de0a3476 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:57.077384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:57.080436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.082287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.083668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:57.085427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:57.090516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:57.191229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1087/2188] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:57 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:57.201003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:57.216031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd147f7-4ce9-48c4-82bb-18102c889478 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:57.430144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:57.433813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.434308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.434810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:57.439616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:57.449429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:57.515097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1102/2189] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:57 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:57.520399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:57.559484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd05df-c5a4-4a7b-9d69-ea4bfd32f6a4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:57.570885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:57.574526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.575185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.575810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:57.577544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:57.595624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:57.674863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1088/2190] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:57.675921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:57.692312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8779d-e781-4500-9175-724b2196e10b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:57.708577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9b5f1-d035-4cda-83f6-043651bafe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:57.709442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b5f1-d035-4cda-83f6-043651bafe4e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.709599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b5f1-d035-4cda-83f6-043651bafe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.709893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b5f1-d035-4cda-83f6-043651bafe4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:57.713317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9b5f1-d035-4cda-83f6-043651bafe4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:57.724224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9b5f1-d035-4cda-83f6-043651bafe4e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:57.762516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1103/2191] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:30:57.765173 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:57.800325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223ab13-9af5-49d5-8b03-858087ef2d0d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:57.825096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:57.828583 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.834983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.836401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] PATH_INFO: `/v3/projects/9b28a91f13764ec5866e44706d30de61/users/6a67fad5af9b499db2afa2679397c256/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:57.858968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:57.881927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b28a91f13764ec5866e44706d30de61, user_id=6a67fad5af9b499db2afa2679397c256, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:57.891247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:57.921734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1089/2192] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:57 2022] PUT /identity/v3/projects/9b28a91f13764ec5866e44706d30de61/users/6a67fad5af9b499db2afa2679397c256/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:57.944942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:57.945447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:57.945943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:58.087974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] MFA Rules not processed for user `6a67fad5af9b499db2afa2679397c256`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:58.090708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:58.339742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1104/2193] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:58.346119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:58.453296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b348c-f439-4172-9919-ddab343c8929 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:58.465138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:58.465874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:58.466463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:58.466679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:58.471306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:58.474899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:58.535218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1090/2194] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:30:58 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:30:58.542671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:58.563044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cd303-0a26-4505-8e96-72c94aeb4fd5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:58.657190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:58.794881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:58.795580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:58.796026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:58.796473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:58.798494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:58.800512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:58.979702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1105/2195] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:30:58 2022] POST /identity/v3/users => generated 300 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:30:58.982416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:59.012551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:59.015613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:59.038862 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a8bdd-1a93-49f6-97b3-636dce00fb67 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:59.041921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82b8f-636f-45f3-b29d-4f3c4d0dda24 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:59.067148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:59.068657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.069170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.069669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] PATH_INFO: `/v3/users/46ad25da636b46fa8fbd8ea00deb2693` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.071959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ad25da636b46fa8fbd8ea00deb2693)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:59.074639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:59.103916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:59.105216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.105738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.106252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.107753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:59.111064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:59.161262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1091/2196] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:59.171957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:59.214696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dce78-faab-4a99-a6b8-224f8354cf05 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:59.227158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ad25da636b46fa8fbd8ea00deb2693'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:59.235392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:59.236361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.236640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.236913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] PATH_INFO: `/v3/projects/1adb09f0b6214a689879b78d659c0ace/users/ac3e7cae02f244a19418cb4d287e084a/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.316151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ad25da636b46fa8fbd8ea00deb2693'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:59.352167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ad25da636b46fa8fbd8ea00deb2693'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:59.354146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ad25da636b46fa8fbd8ea00deb2693'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:59.354888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adb09f0b6214a689879b78d659c0ace, user_id=ac3e7cae02f244a19418cb4d287e084a, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:59.360986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29b02207-7417-4900-88cc-9e972bbad7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ad25da636b46fa8fbd8ea00deb2693'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:30:59.364421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:59.401998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1106/2197] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:30:59 2022] DELETE /identity/v3/users/46ad25da636b46fa8fbd8ea00deb2693 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:59.409641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.410343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.410780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.420563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1092/2198] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:59 2022] PUT /identity/v3/projects/1adb09f0b6214a689879b78d659c0ace/users/ac3e7cae02f244a19418cb4d287e084a/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:59.434114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:59.439135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:59.474942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:30:59.480834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:59.643891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee259a2b-06b9-42ac-92e3-776eb3b6e74d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:59.726304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:59.758054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:59.758394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.759426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.760218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.780215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:59.780215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:59.781446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1107/2199] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:30:59.799925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1093/2200] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:30:59.801652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:59.828965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:59.835231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:59.835543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570647-0f84-4cd4-9513-a263d4d006a9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:59.857024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7064e-90ea-4498-9f26-9f40991384f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:59.858443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7064e-90ea-4498-9f26-9f40991384f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.858734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7064e-90ea-4498-9f26-9f40991384f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.858989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7064e-90ea-4498-9f26-9f40991384f1 admin admin] PATH_INFO: `/v3/projects/1adb09f0b6214a689879b78d659c0ace/users/ac3e7cae02f244a19418cb4d287e084a/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.867362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7064e-90ea-4498-9f26-9f40991384f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adb09f0b6214a689879b78d659c0ace, user_id=ac3e7cae02f244a19418cb4d287e084a, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:59.871591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7064e-90ea-4498-9f26-9f40991384f1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:59.910935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1108/2201] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:30:59 2022] PUT /identity/v3/projects/1adb09f0b6214a689879b78d659c0ace/users/ac3e7cae02f244a19418cb4d287e084a/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:30:59.922683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:59.924158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:59.925046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:59.956150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c570-99ae-4e47-9752-ce0a169be8b8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:00.048872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:00.050762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:00.051292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:00.051801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:00.111254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] MFA Rules not processed for user `ac3e7cae02f244a19418cb4d287e084a`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:00.113446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:00.153041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:00.168295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:00.171090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:00.207416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1094/2202] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:00.314111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1109/2203] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:30:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:00.320984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:00.323099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:00.399350 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:00.407067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356e8da-1bc3-4297-9997-23ad18915397 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:00.426538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:00.428720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:00.429261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:00.431029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:00.474682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:00.479569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:00.531615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1095/2204] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:02.433051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:02.435742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:02.485894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d68104-0d15-4697-b824-6613a3f1fdcd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:02.514217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:02.515207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:02.515499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:02.515782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] PATH_INFO: `/v3/users/bbb32a220826411487abab6cb4bf9f86` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:02.520337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb32a220826411487abab6cb4bf9f86)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:02.528564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:02.603229 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:02.606147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:02.641746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd944d-67ae-439f-8540-de02b9c46b6c service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:02.660796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:02.664028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:02.664378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:02.664614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] PATH_INFO: `/v3/projects/366f74fa3db640e3854cbdeef105ec8c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:02.673406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=366f74fa3db640e3854cbdeef105ec8c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:02.682953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:02.750725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb32a220826411487abab6cb4bf9f86'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.801856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb32a220826411487abab6cb4bf9f86'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.834461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb32a220826411487abab6cb4bf9f86'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.835402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb32a220826411487abab6cb4bf9f86'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.847054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb32a220826411487abab6cb4bf9f86'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.872102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1110/2205] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:02 2022] DELETE /identity/v3/users/bbb32a220826411487abab6cb4bf9f86 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:02.873260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:02.879616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:02.888776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '366f74fa3db640e3854cbdeef105ec8c', 'request_id': 'req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.913357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '366f74fa3db640e3854cbdeef105ec8c', 'request_id': 'req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.937714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '366f74fa3db640e3854cbdeef105ec8c', 'request_id': 'req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.988850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] The token cache is being invalidate because project 366f74fa3db640e3854cbdeef105ec8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:02.989761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:02.996204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-eb99dc77-4dc1-4ff4-a686-c6a5458ec784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.002975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1096/2206] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:02 2022] DELETE /identity/v3/projects/366f74fa3db640e3854cbdeef105ec8c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:03.005968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:03.006799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:03.007507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:03.065401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:03.067194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:03.122384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab9f0-d9ac-4fa7-b7c7-a61bd4b99503 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:03.144301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:03.148122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:03.148926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:03.150020 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] PATH_INFO: `/v3/users/6a67fad5af9b499db2afa2679397c256` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:03.154071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a67fad5af9b499db2afa2679397c256)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:03.161798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:03.188159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1097/2207] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:03.190563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:03.200086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:03.200496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:03.223757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310dfbc-7b5a-4168-aa43-212170209535 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:03.232512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:03.234169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:03.234627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:03.235096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] PATH_INFO: `/v3/users/0fbf4c2c4d88422e9473a02df4058393` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:03.236942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbf4c2c4d88422e9473a02df4058393)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:03.240158 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:03.266191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a67fad5af9b499db2afa2679397c256'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.289005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a67fad5af9b499db2afa2679397c256'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.301951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a67fad5af9b499db2afa2679397c256'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.304929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a67fad5af9b499db2afa2679397c256'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.306042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a67fad5af9b499db2afa2679397c256'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.372659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1111/2208] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:02 2022] DELETE /identity/v3/users/6a67fad5af9b499db2afa2679397c256 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:03.373772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:03.378414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:03.403716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbf4c2c4d88422e9473a02df4058393'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.428888 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbf4c2c4d88422e9473a02df4058393'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.444017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbf4c2c4d88422e9473a02df4058393'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.452413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:03.473604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbf4c2c4d88422e9473a02df4058393'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.474452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-df41a75d-b649-435f-9024-d6ffa0cf1dab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbf4c2c4d88422e9473a02df4058393'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:03.483877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1098/2209] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:03 2022] DELETE /identity/v3/users/0fbf4c2c4d88422e9473a02df4058393 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:03.489655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:03.490356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:03.490891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:03.566719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:03.568253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:03.597195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43dbd-474b-42e0-9052-a560ec007efb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:03.755160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1099/2210] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:03.805454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:03.808404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:03.811255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:03.823055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:03.825433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:03.826028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:03.835525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:03.850202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb818a-c56b-4b3f-93e1-5cbee2cf36c6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:03.886826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaea52e-1333-4409-9dc3-d2eb22fcdac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:03.889291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaea52e-1333-4409-9dc3-d2eb22fcdac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:03.889685 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaea52e-1333-4409-9dc3-d2eb22fcdac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:03.890587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaea52e-1333-4409-9dc3-d2eb22fcdac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:03.898672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:03.904337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:03.925536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaea52e-1333-4409-9dc3-d2eb22fcdac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:03.935207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaea52e-1333-4409-9dc3-d2eb22fcdac0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:03.945081 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1112/2211] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:03.958238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1100/2212] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:04.371197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:04.376595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:04.407955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b7c09-38a7-491f-a189-7e9244f141ac service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:04.441385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:04.445169 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:04.451374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:04.452249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] PATH_INFO: `/v3/projects/297adc38bf7f447e85d6fccfac29817b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:04.456842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=297adc38bf7f447e85d6fccfac29817b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:04.464784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:04.606334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:04.610334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297adc38bf7f447e85d6fccfac29817b', 'request_id': 'req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:04.621437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297adc38bf7f447e85d6fccfac29817b', 'request_id': 'req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:04.637925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297adc38bf7f447e85d6fccfac29817b', 'request_id': 'req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:04.659220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] The token cache is being invalidate because project 297adc38bf7f447e85d6fccfac29817b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:04.659601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:04.660382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b2e9fa23-f4c0-4451-8669-42b8f90c8a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:04.664339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1113/2213] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:04 2022] DELETE /identity/v3/projects/297adc38bf7f447e85d6fccfac29817b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:04.670263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:04.670858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:04.670858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:04.693758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:04.695401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:04.861205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1101/2214] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:04.878273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:04.878806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:04.879233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:04.920106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:04.920940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:04.968042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1114/2215] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:04.969122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.010370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8f1c8-03a7-44f7-b53d-f99446fe839d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:05.018696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:05.018696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:05.019135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:05.019295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:05.020285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:31:05.020655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:05.022437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:05.050895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1102/2216] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:31:05.058703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.085825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c677627-d09c-4936-8a3c-00b6ccef7307 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:05.095420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:05.096583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:05.097340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:05.097975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:05.099456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:05.103871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:05.210664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1115/2217] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:05 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:05.218257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.244358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec4a5-5a1e-4b58-922a-ba823f312749 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:05.444726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.446974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:05.453971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:05.457480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:05.458064 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:05.458543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:05.464146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:05.469327 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:05.472859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89984c9-dda0-4ebe-b55f-e0e33d35d766 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:05.481016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:05.498334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:05.499495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:05.499852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:05.500231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] PATH_INFO: `/v3/projects/9b28a91f13764ec5866e44706d30de61` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:05.510330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b28a91f13764ec5866e44706d30de61)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:05.526694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:05.547949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1103/2218] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:05 2022] POST /identity/v3/users => generated 283 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:05.549588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.552735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:05.590564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f9de-cd9c-403f-87e8-059430f3bd20 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:05.614795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:05.615834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:05.616209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:05.616516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] PATH_INFO: `/v3/projects/41f24044f5f84ce5b3653519d325a92a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:05.619603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f24044f5f84ce5b3653519d325a92a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:05.629302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:05.678401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b28a91f13764ec5866e44706d30de61', 'request_id': 'req-7eb01882-6b08-49f5-80c8-958f443fa460'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.705087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b28a91f13764ec5866e44706d30de61', 'request_id': 'req-7eb01882-6b08-49f5-80c8-958f443fa460'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.736356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b28a91f13764ec5866e44706d30de61', 'request_id': 'req-7eb01882-6b08-49f5-80c8-958f443fa460'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.752829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f24044f5f84ce5b3653519d325a92a', 'request_id': 'req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.763634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f24044f5f84ce5b3653519d325a92a', 'request_id': 'req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.786944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f24044f5f84ce5b3653519d325a92a', 'request_id': 'req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.792581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] The token cache is being invalidate because project 9b28a91f13764ec5866e44706d30de61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:05.792904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.794433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.797601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1116/2219] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:05 2022] DELETE /identity/v3/projects/9b28a91f13764ec5866e44706d30de61 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:05.802454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.804132 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:05.819400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] The token cache is being invalidate because project 41f24044f5f84ce5b3653519d325a92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:05.819852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.820599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:05.823631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1104/2220] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:05 2022] DELETE /identity/v3/projects/41f24044f5f84ce5b3653519d325a92a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:05.825201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:05.826640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:05.978414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc315d9-bfc4-4f31-9b3d-5613b8abc5ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:05.988855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:05.989798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:05.990118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:05.991645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] PATH_INFO: `/v3/users/ac3e7cae02f244a19418cb4d287e084a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:05.996092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3e7cae02f244a19418cb4d287e084a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:05.999512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:06.041621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb01882-6b08-49f5-80c8-958f443fa460 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:06.059803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:06.060513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:06.060792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:06.061067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:06.064099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:06.064099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:06.081168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1117/2221] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:06.086153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:06.089108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:06.102792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3e7cae02f244a19418cb4d287e084a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:06.105580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0b60e-56d1-4632-b95a-ad4a14c99e4b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:06.111206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:06.112241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:06.112705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3e7cae02f244a19418cb4d287e084a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:06.113035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:06.113762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:06.114716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:06.116693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:06.129718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3e7cae02f244a19418cb4d287e084a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:06.139887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1118/2222] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:06.141273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:06.142324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=5,keystone:SELECT=79,keystone:DELETE=16 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:06.159549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3e7cae02f244a19418cb4d287e084a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:06.160565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0a76281d-fa5f-4f47-a7fa-4acfd2d73990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3e7cae02f244a19418cb4d287e084a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:06.170053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94479fbf-5d2a-4267-adfb-638864dbc055 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:06.172938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1105/2223] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:05 2022] DELETE /identity/v3/users/ac3e7cae02f244a19418cb4d287e084a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:06.174778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:06.175739 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:06.176342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:06.176521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] PATH_INFO: `/v3/projects/9457b9fe64c241a1b1bf36eff809d703/users/455bcba0acf240b091dea3efaab15597/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:06.191759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:06.198003 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9457b9fe64c241a1b1bf36eff809d703, user_id=455bcba0acf240b091dea3efaab15597, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:06.201216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:06.255317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1119/2224] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:06 2022] PUT /identity/v3/projects/9457b9fe64c241a1b1bf36eff809d703/users/455bcba0acf240b091dea3efaab15597/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:06.268247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:06.268554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:06.268875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:06.366159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] MFA Rules not processed for user `455bcba0acf240b091dea3efaab15597`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:06.367452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:06.694874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1106/2225] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:06.698792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:06.714904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:07.011641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4812f6-64b8-4e9d-9d14-6dfcc3812f74 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.018967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.020454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.021034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.021587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.023100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.025875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.084126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1120/2226] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:06 2022] POST /identity/v3/projects => generated 367 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:07.085935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.124326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1659-8aa3-467d-996d-fc2210319952 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.214504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.215751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.216039 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.216345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.217590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.221716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.268119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1107/2227] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:07 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:07.272556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.304748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb828643-4a06-487f-a5b0-2de56e711ae4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.306893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.310227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:07.319498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.320224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.320514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.320780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.322254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.324532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.347787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1121/2228] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:07.348995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.363237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862292ab-99d3-4699-9af9-220fbb6f206d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.363657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59898712-d023-4379-a116-d5d50213eba4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.370732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.371847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.372249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.372586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] PATH_INFO: `/v3/projects/1654a0a0b1a2469298ad3b3b38092d2d/users/89398a6375bc40e1b6830d36fe7bbd41/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.379488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.381497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.382010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.382509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] PATH_INFO: `/v3/projects/1adb09f0b6214a689879b78d659c0ace` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.388528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adb09f0b6214a689879b78d659c0ace)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.392852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.407725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1654a0a0b1a2469298ad3b3b38092d2d, user_id=89398a6375bc40e1b6830d36fe7bbd41, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.417956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.450372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1122/2229] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:07 2022] PUT /identity/v3/projects/1654a0a0b1a2469298ad3b3b38092d2d/users/89398a6375bc40e1b6830d36fe7bbd41/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:07.454841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.540803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adb09f0b6214a689879b78d659c0ace', 'request_id': 'req-65102c90-ea3b-4808-a3f3-3e87488d2e09'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:07.564811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adb09f0b6214a689879b78d659c0ace', 'request_id': 'req-65102c90-ea3b-4808-a3f3-3e87488d2e09'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:07.600465 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adb09f0b6214a689879b78d659c0ace', 'request_id': 'req-65102c90-ea3b-4808-a3f3-3e87488d2e09'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:07.606176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:07.631164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] The token cache is being invalidate because project 1adb09f0b6214a689879b78d659c0ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:07.631492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:07.632246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:07.640033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1108/2230] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:07 2022] DELETE /identity/v3/projects/1adb09f0b6214a689879b78d659c0ace => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:31:07.642137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.643925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:07.693682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b752676-f247-4204-9ac8-54e7f04886c7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.773876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102c90-ea3b-4808-a3f3-3e87488d2e09 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.794258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.795316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.795702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.795982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] PATH_INFO: `/v3/users/3c00ab18c6db4a70b972c320319585d2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.799826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c00ab18c6db4a70b972c320319585d2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.814145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.826155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:07.856126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb926c-3c65-4794-9b8d-42382437d70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.856740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb926c-3c65-4794-9b8d-42382437d70f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.857027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb926c-3c65-4794-9b8d-42382437d70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.857396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb926c-3c65-4794-9b8d-42382437d70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.858349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb926c-3c65-4794-9b8d-42382437d70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:07.860041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb926c-3c65-4794-9b8d-42382437d70f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:07.885504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1123/2231] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:07.886002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.886272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.886562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.934847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:07.937555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:07.956583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1124/2232] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:07.957885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:07.967421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:07.985458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e1344-18c8-410e-9bfa-818eb49d04ed None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:07.992137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219337-a641-4ba6-87bf-544e9ac9d1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:07.994009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219337-a641-4ba6-87bf-544e9ac9d1de admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:07.994496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219337-a641-4ba6-87bf-544e9ac9d1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:07.995006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219337-a641-4ba6-87bf-544e9ac9d1de admin admin] PATH_INFO: `/v3/projects/1654a0a0b1a2469298ad3b3b38092d2d/users/89398a6375bc40e1b6830d36fe7bbd41/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:07.999548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219337-a641-4ba6-87bf-544e9ac9d1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1654a0a0b1a2469298ad3b3b38092d2d, user_id=89398a6375bc40e1b6830d36fe7bbd41, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:08.004135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c00ab18c6db4a70b972c320319585d2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:08.005432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219337-a641-4ba6-87bf-544e9ac9d1de admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:08.039600 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c00ab18c6db4a70b972c320319585d2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:08.045703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1125/2233] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:07 2022] PUT /identity/v3/projects/1654a0a0b1a2469298ad3b3b38092d2d/users/89398a6375bc40e1b6830d36fe7bbd41/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:08.047804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c1f47-9008-4dbc-aeef-7284857e49b6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.048532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c1f47-9008-4dbc-aeef-7284857e49b6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.049055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c1f47-9008-4dbc-aeef-7284857e49b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.054933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c00ab18c6db4a70b972c320319585d2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:08.079784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c00ab18c6db4a70b972c320319585d2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:08.080619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c13f7ebf-a6cf-4f00-bb89-71bc96b24af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c00ab18c6db4a70b972c320319585d2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:08.090681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1109/2234] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:07 2022] DELETE /identity/v3/users/3c00ab18c6db4a70b972c320319585d2 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:08.091851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.092155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.092443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.130422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fd9c1f47-9008-4dbc-aeef-7284857e49b6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:08.132228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fd9c1f47-9008-4dbc-aeef-7284857e49b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.178301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] MFA Rules not processed for user `89398a6375bc40e1b6830d36fe7bbd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:08.179882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.331330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1126/2235] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:08.333947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.334512 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.334903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.382149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:08.386769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.411709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1127/2236] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:08.413009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:08.417411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:08.436984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1110/2237] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:08.441670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca6961-5dfb-4571-b1e8-5104d9b993cd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:08.450483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:08.453701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.472895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:08.475473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.478360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.479727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.482827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:31:08.484747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:08.492100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:08.548686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1128/2238] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:31:08.557326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:08.574033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.665824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9893-9d0b-41e5-858c-6ba52ff880b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:08.713415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:08.714178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.714459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.714717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.724988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ae4b-7407-4066-88f7-5e7a2e300bcc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:08.727334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:08.728981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:08.732915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205c80e-8473-4a1b-ba2a-b105af48a46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:08.735114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205c80e-8473-4a1b-ba2a-b105af48a46c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.735674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205c80e-8473-4a1b-ba2a-b105af48a46c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.737152 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205c80e-8473-4a1b-ba2a-b105af48a46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.743483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1111/2239] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:31:08.745383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:08.766151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22d07d-01db-4302-85ed-66c4bc1a5fb0 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:08.770259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205c80e-8473-4a1b-ba2a-b105af48a46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:08.773802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205c80e-8473-4a1b-ba2a-b105af48a46c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:08.777070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:08.778466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.779207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.780156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.781929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:08.785847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:08.802436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1129/2240] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:08.807219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:08.807828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:08.808557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:08.854643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1112/2241] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:08 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:08.854643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:08.861270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.864190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] MFA Rules not processed for user `0d7d57c4231b49b28a606a17e42267e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:08.866551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:08.960204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:09.032389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1130/2242] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:09.033228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:09.088886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00717de7-f724-41f2-abf0-fbade2a98b66 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:09.097988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:09.109386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:09.110866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:09.111507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:09.114663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:09.281110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b8d8a-0a4b-4db1-8e5c-bd835d8912f1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:09.295132 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:09.300315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:09.300957 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:09.301459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:09.309066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:09.314359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:09.315030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:09.316894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:09.342270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1113/2243] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:09.401468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1131/2244] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:09 2022] POST /identity/v3/users => generated 296 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:09.403671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:09.420698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592128eb-6885-41c0-878c-b6024e42d9bb service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:09.433615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:09.437414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:09.437987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:09.438829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:09.442190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:09.445608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:09.470873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1114/2245] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:09.478373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:09.531677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c7b31-0dbe-4be0-adec-c07099dcfc26 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:09.546828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bb594-321b-4b1c-9621-a42df8eac8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:09.548377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb594-321b-4b1c-9621-a42df8eac8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:09.548629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb594-321b-4b1c-9621-a42df8eac8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:09.549102 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb594-321b-4b1c-9621-a42df8eac8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:09.553662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bb594-321b-4b1c-9621-a42df8eac8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:09.558137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bb594-321b-4b1c-9621-a42df8eac8c0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:09.582255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1132/2246] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:09.586458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:09.593840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:09.599980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb032a-41b0-4fc9-ad13-f775f3b9cd51 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:09.619598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:09.620632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:09.621160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:09.621453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] PATH_INFO: `/v3/projects/29be734985f8499a9aed692982600d6e/users/ebd2df94f1734772ab561fb8ca223d94/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:09.648686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29be734985f8499a9aed692982600d6e, user_id=ebd2df94f1734772ab561fb8ca223d94, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:09.653557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:09.693540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1115/2247] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:09 2022] PUT /identity/v3/projects/29be734985f8499a9aed692982600d6e/users/ebd2df94f1734772ab561fb8ca223d94/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:09.768098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:09.768450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:09.768729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:09.863782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] MFA Rules not processed for user `ebd2df94f1734772ab561fb8ca223d94`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:09.871519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:09.969815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:09.974401 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:10.137090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1133/2248] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:10.142154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ea38d-a253-454b-9a29-b9076d5a32b6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:10.151212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:10.168236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:10.172368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:10.172980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:10.173468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] PATH_INFO: `/v3/projects/e153f3946d3e414fb5931013f9f385c5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:10.177658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e153f3946d3e414fb5931013f9f385c5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:10.182399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:10.347782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3bbf1-9038-42a3-b054-7ff6fa5385fb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:10.362926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e153f3946d3e414fb5931013f9f385c5', 'request_id': 'req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:10.379424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e153f3946d3e414fb5931013f9f385c5', 'request_id': 'req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:10.381708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:10.408601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e153f3946d3e414fb5931013f9f385c5', 'request_id': 'req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:10.445323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] The token cache is being invalidate because project e153f3946d3e414fb5931013f9f385c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:10.446402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:10.449534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d7575b3-ad4c-4c4b-b66f-34772f2c7f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:10.452547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1116/2249] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:09 2022] DELETE /identity/v3/projects/e153f3946d3e414fb5931013f9f385c5 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:31:10.458999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6296844-fabf-402a-96e5-20e694ee5889 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:10.459285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6296844-fabf-402a-96e5-20e694ee5889 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:10.459565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6296844-fabf-402a-96e5-20e694ee5889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:10.503689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a6296844-fabf-402a-96e5-20e694ee5889 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:10.506784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a6296844-fabf-402a-96e5-20e694ee5889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:10.525366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:10.526126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:10.526486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:10.526741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:10.533303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:10.713254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:10.715672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:10.747256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1117/2250] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:31:10.749093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:10.749382 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:10.749846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:10.753573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1134/2251] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:31:10.789673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:10.792275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:10.802409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1118/2252] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:10.811669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:10.850901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffe6cb-08ef-4731-ac57-02377459bb95 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:10.871075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:10.872211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:10.873452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:10.874088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:10.875034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:31:10.875499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:10.877385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:10.893373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1135/2253] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:31:10.900325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:10.905744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:10.920611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef70e3a-db54-4628-8af1-60f73948a73b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:10.947555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:10.951455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:10.953718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:10.955128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:10.955921 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:10.967488 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:11.061493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1119/2254] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:10 2022] POST /identity/v3/projects => generated 397 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:31:11.066964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:11.089472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322719d-a43c-40ca-a954-affa3d8d8a43 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:11.308099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:11.308814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:11.309092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:11.309366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:11.310086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:11.319786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:11.421815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1136/2255] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:11 2022] POST /identity/v3/users => generated 299 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:11.427377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:11.462389 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bbc10-658d-4a50-b89c-48df113bb5bd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:11.480517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:11.481320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:11.481609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:11.481889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:11.482713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:11.484434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:11.518755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1120/2256] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:11.520028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:11.536107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c7c65-2613-4cd0-8eb6-dee19c749010 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:11.551718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5a597-e8c3-4cb1-97d2-f5cb71e4d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:11.552468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5a597-e8c3-4cb1-97d2-f5cb71e4d262 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:11.553080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5a597-e8c3-4cb1-97d2-f5cb71e4d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:11.553249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5a597-e8c3-4cb1-97d2-f5cb71e4d262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:11.554012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5a597-e8c3-4cb1-97d2-f5cb71e4d262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:11.558173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5a597-e8c3-4cb1-97d2-f5cb71e4d262 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:11.574044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1137/2257] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:11.581140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:11.608508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0f342-ce78-4082-9697-b65a6ea35cde admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:11.624616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:11.626555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:11.627160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:11.627800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] PATH_INFO: `/v3/projects/24805c96ab48499c860b71c28d1fd8fd/users/b119dc91fe19459b813f2d9dff02d874/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:11.667061 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=24805c96ab48499c860b71c28d1fd8fd, user_id=b119dc91fe19459b813f2d9dff02d874, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:11.671388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:11.718539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1121/2258] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:11 2022] PUT /identity/v3/projects/24805c96ab48499c860b71c28d1fd8fd/users/b119dc91fe19459b813f2d9dff02d874/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:11.729348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dda5b-9af5-4a96-8308-510eb5909017 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:11.729932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dda5b-9af5-4a96-8308-510eb5909017 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:11.730458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dda5b-9af5-4a96-8308-510eb5909017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:11.849760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2e1dda5b-9af5-4a96-8308-510eb5909017 None None] MFA Rules not processed for user `b119dc91fe19459b813f2d9dff02d874`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:11.853207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2e1dda5b-9af5-4a96-8308-510eb5909017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:11.927744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:12.016129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1138/2259] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:12.032312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:12.035270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:12.197934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a126b-2100-427b-ba74-7897831ee5db admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:12.205489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:12.206548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:12.207379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:12.208006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:12.226108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:12.227949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:12.239142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1122/2260] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:12.968887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:12.969502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:12.969842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:13.050859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:13.053351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:13.233031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1139/2261] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:31:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:31:13.263070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:13.265444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:13.513616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933dd7f-856a-4b9d-8b87-7a1f2447ba78 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:13.522326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a981eae-7c7a-4207-b769-ac06943f2b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:13.523021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a981eae-7c7a-4207-b769-ac06943f2b7e service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:13.523303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a981eae-7c7a-4207-b769-ac06943f2b7e service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:13.523682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a981eae-7c7a-4207-b769-ac06943f2b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:13.541075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a981eae-7c7a-4207-b769-ac06943f2b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:13.543026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a981eae-7c7a-4207-b769-ac06943f2b7e service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:13.557033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1123/2262] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:31:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:31:14.493952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:14.495820 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:14.724859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e62e7-6410-449a-88cd-0a877a47ffe0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:14.730669 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:14.731878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:14.732357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:14.732805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] PATH_INFO: `/v3/users/455bcba0acf240b091dea3efaab15597` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:14.734871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] RBAC: Authorizing `identity:delete_user(user_id=455bcba0acf240b091dea3efaab15597)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:14.738083 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:14.832759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455bcba0acf240b091dea3efaab15597'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:14.848672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:14.868781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455bcba0acf240b091dea3efaab15597'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:14.878908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455bcba0acf240b091dea3efaab15597'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:14.879765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455bcba0acf240b091dea3efaab15597'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:14.880528 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455bcba0acf240b091dea3efaab15597'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:14.912668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1140/2263] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:14 2022] DELETE /identity/v3/users/455bcba0acf240b091dea3efaab15597 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:14.921144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:14.922316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:14.923611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:14.966256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:14.969344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:15.121700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:15.199436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1124/2264] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:15.208441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:15.210804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:15.369719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815221ab-e3ff-4920-b09c-15aac107951b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:15.387520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:15.388757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:15.389391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:15.390101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:15.408215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:15.410599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:15.440604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1141/2265] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:16.068707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:16.073563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:16.132317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e19905-ce5b-40d8-a17e-cd6dc56e3c93 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:16.147522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:16.150712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:16.151180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:16.151714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] PATH_INFO: `/v3/projects/9457b9fe64c241a1b1bf36eff809d703` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:16.153566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9457b9fe64c241a1b1bf36eff809d703)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:16.159024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:16.376817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9457b9fe64c241a1b1bf36eff809d703', 'request_id': 'req-07bc128b-cff9-44f6-bc7f-90f916c77a02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.425352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9457b9fe64c241a1b1bf36eff809d703', 'request_id': 'req-07bc128b-cff9-44f6-bc7f-90f916c77a02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.449117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9457b9fe64c241a1b1bf36eff809d703', 'request_id': 'req-07bc128b-cff9-44f6-bc7f-90f916c77a02'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.490063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] The token cache is being invalidate because project 9457b9fe64c241a1b1bf36eff809d703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:16.490753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.493509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.502079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:16.514712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1125/2266] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:16 2022] DELETE /identity/v3/projects/9457b9fe64c241a1b1bf36eff809d703 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:16.535398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afeed3-c3eb-4942-be64-7a6d06724961 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:16.745014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:16.746342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:16.746867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:16.747564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] PATH_INFO: `/v3/users/89398a6375bc40e1b6830d36fe7bbd41` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:16.749451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=89398a6375bc40e1b6830d36fe7bbd41)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:16.753761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:16.857579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89398a6375bc40e1b6830d36fe7bbd41'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.888488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89398a6375bc40e1b6830d36fe7bbd41'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.908982 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89398a6375bc40e1b6830d36fe7bbd41'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.911636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89398a6375bc40e1b6830d36fe7bbd41'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.922356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df6c1a8-5e70-44d9-8c9c-83e23b8ff1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89398a6375bc40e1b6830d36fe7bbd41'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:16.929595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:16.971779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1142/2267] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:16 2022] DELETE /identity/v3/users/89398a6375bc40e1b6830d36fe7bbd41 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:17.777696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:17.779889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:17.807261 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc128b-cff9-44f6-bc7f-90f916c77a02 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:18.031130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:18.033557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.034197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.035086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] PATH_INFO: `/v3/projects/1654a0a0b1a2469298ad3b3b38092d2d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.037615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1654a0a0b1a2469298ad3b3b38092d2d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:18.042001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:18.202385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1654a0a0b1a2469298ad3b3b38092d2d', 'request_id': 'req-76e5bec1-4688-4f76-852c-8795dcfd2f4c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:18.217725 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1654a0a0b1a2469298ad3b3b38092d2d', 'request_id': 'req-76e5bec1-4688-4f76-852c-8795dcfd2f4c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:18.237641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1654a0a0b1a2469298ad3b3b38092d2d', 'request_id': 'req-76e5bec1-4688-4f76-852c-8795dcfd2f4c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:18.240756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:18.290969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] The token cache is being invalidate because project 1654a0a0b1a2469298ad3b3b38092d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:18.292270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:18.293846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-76e5bec1-4688-4f76-852c-8795dcfd2f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:18.298390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1126/2268] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:17 2022] DELETE /identity/v3/projects/1654a0a0b1a2469298ad3b3b38092d2d => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:18.303737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.304361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.304361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.342071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:18.345286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:18.510433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1143/2269] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:18.520865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.521144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.521493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.546395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:18.547644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:18.558175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1127/2270] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:18.563228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:18.583236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37126c-21ef-42a0-a089-951929759cc5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:18.594068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:18.595629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.596212 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.596782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.599194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:31:18.599761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:18.604578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:18.634186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1144/2271] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:31:18.638831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:18.652631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78544c02-45c7-47b2-884e-7f12365fbebc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:18.658033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:18.659022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.659380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.659748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.661128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:18.663268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:18.705189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1128/2272] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:18 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:31:18.711457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:18.734160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf0c55-7218-4802-be40-88c3af982bb1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:18.886923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:18.888371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.888902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.889405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.890798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:18.894581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:18.957399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1145/2273] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:18 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:18.963654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:18.975634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d543-fdb3-4d9b-bb62-3f9fb062f5fa admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:18.987186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:18.988136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:18.988546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:18.988934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:18.989995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:18.992095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:19.009287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1129/2274] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:19.014398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:19.073563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee5ab-0f16-4be1-b594-c57553b32fb5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:19.096185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c2366-7c15-445c-bded-53b4441d511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:19.097390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c2366-7c15-445c-bded-53b4441d511f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:19.097901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c2366-7c15-445c-bded-53b4441d511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:19.098403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c2366-7c15-445c-bded-53b4441d511f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:19.099915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c2366-7c15-445c-bded-53b4441d511f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:19.104597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c2366-7c15-445c-bded-53b4441d511f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:19.130989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1146/2275] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:19.139830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:19.154218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754e1ca-d9a1-4715-9359-a6d5b3345add admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:19.162899 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:19.164028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:19.164451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:19.166203 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] PATH_INFO: `/v3/projects/c26f410423a1440cb4a90acc21b98e4d/users/5d1d6572f84647829f394c63db940207/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:19.185397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26f410423a1440cb4a90acc21b98e4d, user_id=5d1d6572f84647829f394c63db940207, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:19.188444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:19.211096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1130/2276] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:19 2022] PUT /identity/v3/projects/c26f410423a1440cb4a90acc21b98e4d/users/5d1d6572f84647829f394c63db940207/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:31:19.235245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:19.235780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:19.236689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:19.324641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] MFA Rules not processed for user `5d1d6572f84647829f394c63db940207`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:19.327351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:19.489643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:19.619660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:19.619951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1147/2277] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:19.772618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a842be-5dfa-479a-b6b5-12f220a2617a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:19.777692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69ff2-3aad-415a-8cb5-f4269f69e192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:19.778573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ff2-3aad-415a-8cb5-f4269f69e192 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:19.778884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ff2-3aad-415a-8cb5-f4269f69e192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:19.779166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ff2-3aad-415a-8cb5-f4269f69e192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:19.780978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69ff2-3aad-415a-8cb5-f4269f69e192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:19.783696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69ff2-3aad-415a-8cb5-f4269f69e192 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:19.847116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:19.856493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1131/2278] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:19 2022] POST /identity/v3/projects => generated 393 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:19.889564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5d10e-0f18-497c-9818-9bcaa271e0ba None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:20.141224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:20.141971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.142277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.142554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.143301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:20.145954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:20.170317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.171211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.171688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.211159 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1148/2279] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:19 2022] POST /identity/v3/users => generated 303 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:20.213838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:20.216460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:20.220958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:20.238928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc49ff-9a93-410b-80e3-f4c67d5a411a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:20.254027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:20.255112 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.255399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.257092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.258229 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:20.265747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:20.306531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1149/2280] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:20.307443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:20.331915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b3e0c-2e74-4be6-aa52-51dcd906a6d2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:20.347922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:20.348952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.349238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.350180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] PATH_INFO: `/v3/projects/843cf289642b4065a84b0ae9ad931d0f/users/9e8fbe2d84cd4934b376560e54d9660d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.377872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1132/2281] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:31:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 18:31:20.405288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] RBAC: Authorizing `identity:create_grant(project_id=843cf289642b4065a84b0ae9ad931d0f, user_id=9e8fbe2d84cd4934b376560e54d9660d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:20.407465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:20.442611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1150/2282] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:20 2022] PUT /identity/v3/projects/843cf289642b4065a84b0ae9ad931d0f/users/9e8fbe2d84cd4934b376560e54d9660d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:20.446055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:20.458176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:20.467472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:20.472452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2a9e3-bb33-453a-aa4c-90e3dfd05e32 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:20.529905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:20.723059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bf5bb-8679-4534-a898-50d2ae91601e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:20.738480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:20.739597 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.740125 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.740632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.742044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:20.747666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:20.748935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:20.751330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.751822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.752347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.792967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1133/2283] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:31:20.797859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:20.841505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f761b5d-2142-4006-bcc5-7aa1d9f5a492 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:20.859606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c7e37-b7db-4016-970a-a2846643531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:20.861177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c7e37-b7db-4016-970a-a2846643531e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.861983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c7e37-b7db-4016-970a-a2846643531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.862464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c7e37-b7db-4016-970a-a2846643531e admin admin] PATH_INFO: `/v3/projects/843cf289642b4065a84b0ae9ad931d0f/users/9e8fbe2d84cd4934b376560e54d9660d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.873846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c7e37-b7db-4016-970a-a2846643531e admin admin] RBAC: Authorizing `identity:create_grant(project_id=843cf289642b4065a84b0ae9ad931d0f, user_id=9e8fbe2d84cd4934b376560e54d9660d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:20.879628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c7e37-b7db-4016-970a-a2846643531e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:20.881738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:20.917649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:20.922041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:20.929318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1134/2284] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:20 2022] PUT /identity/v3/projects/843cf289642b4065a84b0ae9ad931d0f/users/9e8fbe2d84cd4934b376560e54d9660d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:31:20.931768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60057-66c6-4f22-aa9d-c75312304819 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:20.932159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60057-66c6-4f22-aa9d-c75312304819 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:20.932429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60057-66c6-4f22-aa9d-c75312304819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:20.949390 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1151/2285] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:31:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:31:21.008988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8fa60057-66c6-4f22-aa9d-c75312304819 None None] MFA Rules not processed for user `9e8fbe2d84cd4934b376560e54d9660d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:21.013407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8fa60057-66c6-4f22-aa9d-c75312304819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:21.272314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1135/2286] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:21.290582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:21.293918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:21.630124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:21.630732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:21.631191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:21.670790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] MFA Rules not processed for user `512f01cca14b447c8f6c9eebbd828eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:21.672671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:21.702500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944f9d4-b50f-4735-b9ff-1b357f0025ac service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:21.832714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1136/2287] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:21.865931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:21.867453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:21.868231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:21.869088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:21.905384 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:21.908559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:21.942340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1152/2288] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:22.048781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:22.066464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:22.140950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14cc32-9d90-4b78-b212-24046130c515 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:22.162959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:22.165234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:22.168802 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:22.169985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:22.314705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:22.449880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:22.452877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:22.496270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1137/2289] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:27.426533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:27.431033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:27.974874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefb4d9-b517-442a-bc6b-c29a930c6195 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:27.989406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:27.994626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:27.995073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:27.995454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:28.001329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:28.071902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:28.086761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec02d0-07c7-46c7-9a29-4f7d1b2c34ce service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:28.127057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1153/2290] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:29.456473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:29.458982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:29.837877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46d7cb-5b61-4660-b779-83473f461260 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:29.851076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:29.864732 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:29.865290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:29.865984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] PATH_INFO: `/v3/users/b119dc91fe19459b813f2d9dff02d874` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:29.881343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b119dc91fe19459b813f2d9dff02d874)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:29.881711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:30.117367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b119dc91fe19459b813f2d9dff02d874'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:30.152564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b119dc91fe19459b813f2d9dff02d874'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:30.168837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b119dc91fe19459b813f2d9dff02d874'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:30.209513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b119dc91fe19459b813f2d9dff02d874'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:30.210299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b119dc91fe19459b813f2d9dff02d874'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:30.229571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1138/2291] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:29 2022] DELETE /identity/v3/users/b119dc91fe19459b813f2d9dff02d874 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:30.234322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:30.235254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:30.235254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:30.292025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:30.293560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:30.322687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:30.548504 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1154/2292] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:30.573712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:30.575530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:30.798735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:30.812517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc10dc4-b990-4de2-82d9-8decccbef1bd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:30.824238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:30.825675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:30.834454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:30.834976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:30.973736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:30.975923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:31.016472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1139/2293] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:31.900579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:31.903041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:31.947973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768b41-fa4c-47b0-b561-15d3c5037613 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:31.983705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:31.984941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:31.985285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:31.985555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] PATH_INFO: `/v3/projects/24805c96ab48499c860b71c28d1fd8fd` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:31.987499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24805c96ab48499c860b71c28d1fd8fd)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:31.994368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:32.130680 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24805c96ab48499c860b71c28d1fd8fd', 'request_id': 'req-54b0e0a3-da77-4b06-b73c-23185706d492'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.149497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24805c96ab48499c860b71c28d1fd8fd', 'request_id': 'req-54b0e0a3-da77-4b06-b73c-23185706d492'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.166661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24805c96ab48499c860b71c28d1fd8fd', 'request_id': 'req-54b0e0a3-da77-4b06-b73c-23185706d492'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.207491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] The token cache is being invalidate because project 24805c96ab48499c860b71c28d1fd8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:32.208306 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.209286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-54b0e0a3-da77-4b06-b73c-23185706d492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.212267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1155/2294] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:31 2022] DELETE /identity/v3/projects/24805c96ab48499c860b71c28d1fd8fd => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:32.291617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:32.300370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:32.371984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e613-a508-4c92-a2d5-56854d118229 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:32.646331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:32.647178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:32.647447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:32.647718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] PATH_INFO: `/v3/users/5d1d6572f84647829f394c63db940207` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:32.658655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1d6572f84647829f394c63db940207)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:32.661160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:32.783155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1d6572f84647829f394c63db940207'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.820090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1d6572f84647829f394c63db940207'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.830447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1d6572f84647829f394c63db940207'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.864138 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1d6572f84647829f394c63db940207'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.865041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1d6572f84647829f394c63db940207'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:32.878708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1140/2295] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:32 2022] DELETE /identity/v3/users/5d1d6572f84647829f394c63db940207 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:32.881774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:32.882718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:32.885893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:32.933483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:32.938992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:33.132568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1156/2296] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:33.145815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:33.147816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:33.345666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73467804-9edd-4143-9376-1b9b5a1be9f3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:33.357502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:33.358847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:33.359320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:33.359800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:33.380995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:33.385182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:33.407439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1141/2297] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:34.229075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:34.234307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:34.269688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443949e-a2f8-4411-87d7-623c779d107a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:34.285968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:34.287977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:34.289016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:34.289747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] PATH_INFO: `/v3/projects/c26f410423a1440cb4a90acc21b98e4d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:34.293227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26f410423a1440cb4a90acc21b98e4d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:34.298652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:34.460210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:34.489907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26f410423a1440cb4a90acc21b98e4d', 'request_id': 'req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.528168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26f410423a1440cb4a90acc21b98e4d', 'request_id': 'req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.538422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26f410423a1440cb4a90acc21b98e4d', 'request_id': 'req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.562347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] The token cache is being invalidate because project c26f410423a1440cb4a90acc21b98e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:34.562751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.563619 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.565759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1157/2298] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:34 2022] DELETE /identity/v3/projects/c26f410423a1440cb4a90acc21b98e4d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:34.573129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:34.614892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0cc66-bb6f-4eb9-8e61-b6bdb2025129 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:34.645784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:34.833610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:34.835556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:34.837053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:34.837551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] PATH_INFO: `/v3/users/9e8fbe2d84cd4934b376560e54d9660d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:34.839838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8fbe2d84cd4934b376560e54d9660d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:34.843376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:34.958769 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8fbe2d84cd4934b376560e54d9660d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.976698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8fbe2d84cd4934b376560e54d9660d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:34.991459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8fbe2d84cd4934b376560e54d9660d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:35.042965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8fbe2d84cd4934b376560e54d9660d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:35.043745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c817408f-6152-4cba-bf1c-0d2eb43b4424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8fbe2d84cd4934b376560e54d9660d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:35.057946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1142/2299] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:34 2022] DELETE /identity/v3/users/9e8fbe2d84cd4934b376560e54d9660d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:35.874590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:35.878880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:35.910823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d07e40-1b8d-4fa1-bfca-e4bc9a2db767 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:36.094440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:36.095421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:36.095709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:36.095989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] PATH_INFO: `/v3/users/ebd2df94f1734772ab561fb8ca223d94` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:36.102074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd2df94f1734772ab561fb8ca223d94)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:36.104392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:36.215795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd2df94f1734772ab561fb8ca223d94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.241874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd2df94f1734772ab561fb8ca223d94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.255072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd2df94f1734772ab561fb8ca223d94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.256109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd2df94f1734772ab561fb8ca223d94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.256762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd2df94f1734772ab561fb8ca223d94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.286491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1158/2300] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:35 2022] DELETE /identity/v3/users/ebd2df94f1734772ab561fb8ca223d94 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:36.288796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:36.290154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:36.290380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:36.328941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:36.331598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:36.443068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:36.449552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:36.465999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1143/2301] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:36.467211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:36.474828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:36.550302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def6bed-52d1-4247-b222-c25c71147f6f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:36.563574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:36.565328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:36.565836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:36.566395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] PATH_INFO: `/v3/projects/843cf289642b4065a84b0ae9ad931d0f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:36.569664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=843cf289642b4065a84b0ae9ad931d0f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:36.575420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:36.653005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:36.663109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:36.698918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb880-58ca-4c5f-8c1b-36aa0c76777c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:36.705726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843cf289642b4065a84b0ae9ad931d0f', 'request_id': 'req-e3838028-ebb2-4bce-b1b3-13dc9234d4be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.724265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843cf289642b4065a84b0ae9ad931d0f', 'request_id': 'req-e3838028-ebb2-4bce-b1b3-13dc9234d4be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.738668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843cf289642b4065a84b0ae9ad931d0f', 'request_id': 'req-e3838028-ebb2-4bce-b1b3-13dc9234d4be'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.782993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] The token cache is being invalidate because project 843cf289642b4065a84b0ae9ad931d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:36.783508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.791182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e3838028-ebb2-4bce-b1b3-13dc9234d4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:36.812915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1144/2302] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:36 2022] DELETE /identity/v3/projects/843cf289642b4065a84b0ae9ad931d0f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:36.815348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a84224-2ddb-4379-b012-93f292067785 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:36.815846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a84224-2ddb-4379-b012-93f292067785 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:36.816387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a84224-2ddb-4379-b012-93f292067785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:36.870095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:36.870717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:36.871075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:36.871469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:36.873984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:36.883932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c1a84224-2ddb-4379-b012-93f292067785 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:36.887503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c1a84224-2ddb-4379-b012-93f292067785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:37.098813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1145/2303] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:37.116358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.117374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.118582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.177879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:37.180560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:37.180963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.183612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.200553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1159/2304] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:31:37.205201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1146/2305] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:37.215480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.263138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15f71d-144e-47a6-ad2e-6283681d647f service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:37.298923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:37.300933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.301269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.301585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.302399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:31:37.302737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.304654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.351355 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1160/2306] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:31:37.360631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.388630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382b9a-8449-49b9-bfee-e7a4dcaa9cdb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:37.403291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:37.405219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.406100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.406756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.408621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.413424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.481176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1147/2307] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:31:37 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:37.486910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.501784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20aa17-5ea5-4251-bc0c-91f25e05611d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:37.670883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:37.672353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.673091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.673562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.675139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.679395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.746907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1161/2308] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:31:37 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:31:37.753496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.774901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a778861-de9a-4824-8bdd-d9277af5ba02 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:37.784592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:37.786021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.786584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.787151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.788615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.790791 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.815787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1148/2309] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:31:37.818902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.844275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.854533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:37.863872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2dcaf-93c3-4727-87bc-0e7311348e82 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:37.875942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:37.878162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.880114 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.881084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.884314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.890052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.918930 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1162/2310] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:31:37.928688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:37.934762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c055284-e4e5-43f2-a397-c04e521d5e68 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:37.950526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:37.951973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:37.952326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:37.952577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] PATH_INFO: `/v3/projects/29be734985f8499a9aed692982600d6e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:37.954077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29be734985f8499a9aed692982600d6e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:37.971834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:37.994538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42953e85-0c0f-4764-8a91-59b0db31642b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:38.018602 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:38.019645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:38.025319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:38.025835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] PATH_INFO: `/v3/projects/318a5d4f4e7d4efca8e2f0b20fb23f28/users/46f73401da024e0eb97fd904cbf2ba35/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:38.078431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:38.092277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29be734985f8499a9aed692982600d6e', 'request_id': 'req-e04d74b8-752e-4b13-8f4d-4713494b37c1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:38.094818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=318a5d4f4e7d4efca8e2f0b20fb23f28, user_id=46f73401da024e0eb97fd904cbf2ba35, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:38.100761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:38.110855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29be734985f8499a9aed692982600d6e', 'request_id': 'req-e04d74b8-752e-4b13-8f4d-4713494b37c1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:38.120815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29be734985f8499a9aed692982600d6e', 'request_id': 'req-e04d74b8-752e-4b13-8f4d-4713494b37c1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:38.159021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:38.164234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] The token cache is being invalidate because project 29be734985f8499a9aed692982600d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:38.164608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:38.165890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e04d74b8-752e-4b13-8f4d-4713494b37c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:38.176171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1149/2311] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:37 2022] DELETE /identity/v3/projects/29be734985f8499a9aed692982600d6e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:38.184452 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1163/2312] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:31:37 2022] PUT /identity/v3/projects/318a5d4f4e7d4efca8e2f0b20fb23f28/users/46f73401da024e0eb97fd904cbf2ba35/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:38.190792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478063f-6742-4af3-a9e2-64865e9cfffa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:38.191398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478063f-6742-4af3-a9e2-64865e9cfffa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:38.191684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478063f-6742-4af3-a9e2-64865e9cfffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:38.285309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0478063f-6742-4af3-a9e2-64865e9cfffa None None] MFA Rules not processed for user `46f73401da024e0eb97fd904cbf2ba35`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:38.287003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0478063f-6742-4af3-a9e2-64865e9cfffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:38.515215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1150/2313] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:38.531995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:38.535770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:38.781805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c5f9-a0c3-4c60-975f-66fc7c21a0bb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:38.790044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b621d-1636-4fab-96db-ca3ba596bf6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:38.792241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b621d-1636-4fab-96db-ca3ba596bf6b service glance] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:38.793056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b621d-1636-4fab-96db-ca3ba596bf6b service glance] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:38.793682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b621d-1636-4fab-96db-ca3ba596bf6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:38.814211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b621d-1636-4fab-96db-ca3ba596bf6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:38.818642 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b621d-1636-4fab-96db-ca3ba596bf6b service glance] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:38.842067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1164/2314] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:45.810714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcf829-937c-4759-9cb1-af35cf6c8cc0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:45.812171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcf829-937c-4759-9cb1-af35cf6c8cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:45.812708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcf829-937c-4759-9cb1-af35cf6c8cc0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:45.815207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1151/2315] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:31:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:31:45.827687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62940a-d84d-47ee-b8fb-d5db3b639e50 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:45.828789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62940a-d84d-47ee-b8fb-d5db3b639e50 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:45.829365 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62940a-d84d-47ee-b8fb-d5db3b639e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:45.922611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3a62940a-d84d-47ee-b8fb-d5db3b639e50 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:45.924918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3a62940a-d84d-47ee-b8fb-d5db3b639e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:46.129586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1165/2316] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:31:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:46.144340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcf829-937c-4759-9cb1-af35cf6c8cc0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:46.147137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d1dcf829-937c-4759-9cb1-af35cf6c8cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:46.177096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcf829-937c-4759-9cb1-af35cf6c8cc0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:46.182243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88be65d-3fa0-4515-9181-fbf224c1dc6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:46.183344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88be65d-3fa0-4515-9181-fbf224c1dc6e service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.183806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88be65d-3fa0-4515-9181-fbf224c1dc6e service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.184135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88be65d-3fa0-4515-9181-fbf224c1dc6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.202425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88be65d-3fa0-4515-9181-fbf224c1dc6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:46.205551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88be65d-3fa0-4515-9181-fbf224c1dc6e service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:46.229159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1152/2317] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:46.324038 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140dcb5-725f-4205-b7de-22d605d8f15b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.324442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140dcb5-725f-4205-b7de-22d605d8f15b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.324724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140dcb5-725f-4205-b7de-22d605d8f15b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.326151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1166/2318] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:31:46.331738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78580-14d8-47c9-9b48-4fb605068dd0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.332020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78580-14d8-47c9-9b48-4fb605068dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.332351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78580-14d8-47c9-9b48-4fb605068dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.354580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e2c78580-14d8-47c9-9b48-4fb605068dd0 None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:46.355833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e2c78580-14d8-47c9-9b48-4fb605068dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:46.444636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1153/2319] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:31:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:31:46.457399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140dcb5-725f-4205-b7de-22d605d8f15b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:46.460794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f140dcb5-725f-4205-b7de-22d605d8f15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:46.499108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140dcb5-725f-4205-b7de-22d605d8f15b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:46.505431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debe0ed-0f29-470c-bf81-ded6d8e616f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:46.506405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debe0ed-0f29-470c-bf81-ded6d8e616f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.506898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debe0ed-0f29-470c-bf81-ded6d8e616f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.507393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debe0ed-0f29-470c-bf81-ded6d8e616f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.522842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debe0ed-0f29-470c-bf81-ded6d8e616f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:46.525509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debe0ed-0f29-470c-bf81-ded6d8e616f2 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:46.536214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1167/2320] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:31:46.721428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd9b77-8179-4647-80eb-1ba5851f177f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.721806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd9b77-8179-4647-80eb-1ba5851f177f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.722043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd9b77-8179-4647-80eb-1ba5851f177f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.723798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1154/2321] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:31:46.732769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3bd45-80ba-4ce6-a46a-8f210b0abd31 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.733476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3bd45-80ba-4ce6-a46a-8f210b0abd31 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.733917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3bd45-80ba-4ce6-a46a-8f210b0abd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.760566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2cf3bd45-80ba-4ce6-a46a-8f210b0abd31 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:46.762023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2cf3bd45-80ba-4ce6-a46a-8f210b0abd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:46.775787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1168/2322] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:31:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:46.793103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd9b77-8179-4647-80eb-1ba5851f177f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:46.811789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd9b77-8179-4647-80eb-1ba5851f177f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:46.818052 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:46.819280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:46.819846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:46.821208 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:46.842154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:46.845178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:46.866591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1155/2323] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:31:50.439393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:50.439986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:50.439986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:50.463440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:50.525673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:50.527514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:50.667146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1169/2324] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:31:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:50.680666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:50.682349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:50.798661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:50.820403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be0d08-19ed-4f43-ab02-f268f82bb8dd service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:50.827681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37a556-4892-4407-892a-424e65c0fd64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:50.828467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a556-4892-4407-892a-424e65c0fd64 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:50.828849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a556-4892-4407-892a-424e65c0fd64 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:50.829574 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a556-4892-4407-892a-424e65c0fd64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:50.846391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37a556-4892-4407-892a-424e65c0fd64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:50.847927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37a556-4892-4407-892a-424e65c0fd64 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:50.859207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1156/2325] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:31:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:31:58.694261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:58.696715 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:58.791149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f2ad5-1855-447a-9322-4417f558a76d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:58.796942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:58.798021 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:58.798306 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:58.798609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] PATH_INFO: `/v3/users/46f73401da024e0eb97fd904cbf2ba35` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:58.799941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f73401da024e0eb97fd904cbf2ba35)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:58.801941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:58.894580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f73401da024e0eb97fd904cbf2ba35'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:58.911295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f73401da024e0eb97fd904cbf2ba35'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:58.923209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f73401da024e0eb97fd904cbf2ba35'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:58.924035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f73401da024e0eb97fd904cbf2ba35'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:58.924711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f73401da024e0eb97fd904cbf2ba35'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:58.970519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1170/2326] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:31:58 2022] DELETE /identity/v3/users/46f73401da024e0eb97fd904cbf2ba35 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:58.977875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:58.979701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:58.980615 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:59.004666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:31:59.005939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:59.112644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1157/2327] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:31:59.124519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:59.125325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:59.142315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:31:59.246293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778991a5-65f4-4f3c-9c73-535f2d91398f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:59.250483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6205-0e76-451d-98a4-ee0418533657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:59.251219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6205-0e76-451d-98a4-ee0418533657 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:59.251625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6205-0e76-451d-98a4-ee0418533657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:59.251942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6205-0e76-451d-98a4-ee0418533657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:59.263663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6205-0e76-451d-98a4-ee0418533657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:59.265201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6205-0e76-451d-98a4-ee0418533657 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:59.278938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1171/2328] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:31:59.676098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:31:59.678105 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:31:59.693619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1f816-4b79-436d-b76a-48e3a44a0810 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:31:59.702128 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:31:59.703956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:59.704580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:59.705192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] PATH_INFO: `/v3/projects/318a5d4f4e7d4efca8e2f0b20fb23f28` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:31:59.707646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] RBAC: Authorizing `identity:delete_project(project_id=318a5d4f4e7d4efca8e2f0b20fb23f28)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:31:59.717289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:31:59.922857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318a5d4f4e7d4efca8e2f0b20fb23f28', 'request_id': 'req-3935d1db-c191-4b89-a68e-e6c9b58a784f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:59.935942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318a5d4f4e7d4efca8e2f0b20fb23f28', 'request_id': 'req-3935d1db-c191-4b89-a68e-e6c9b58a784f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:59.959701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318a5d4f4e7d4efca8e2f0b20fb23f28', 'request_id': 'req-3935d1db-c191-4b89-a68e-e6c9b58a784f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:59.979779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] The token cache is being invalidate because project 318a5d4f4e7d4efca8e2f0b20fb23f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:31:59.980951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:59.980951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3935d1db-c191-4b89-a68e-e6c9b58a784f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:31:59.983026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1158/2329] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:31:59 2022] DELETE /identity/v3/projects/318a5d4f4e7d4efca8e2f0b20fb23f28 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:31:59.987893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:31:59.988558 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:31:59.988652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.017254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:00.018825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:00.136776 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1172/2330] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:32:00.146709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.146709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.146709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.167918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:00.169330 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:00.182869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1159/2331] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:00.187124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.212091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a21da-6a81-4e73-9399-df9ba816cb7c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.227596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.232088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.232088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.232088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.232088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:32:00.232088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.232920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.252096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1173/2332] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:32:00.254409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.266243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00f8bc-6943-4778-9015-3c67bc2a8c5c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.281837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.282543 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.282829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.283120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.283811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.285834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.332133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1160/2333] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:32:00 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:32:00.337119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.349682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acc28-871f-4b3e-bf90-798a9d854516 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.436969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.438107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.438107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.438430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.439628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.442645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.490129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1174/2334] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:32:00 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:32:00.491893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.512226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40ca80-9b85-42bd-bd4e-0a20190b4f61 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.520107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.525202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.525598 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.525922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.526890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.530056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.551079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1161/2335] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:32:00.556594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.568516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0ba32-fbef-4da5-8cf5-0196008657b7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.574206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91efaa-b444-42fb-ad1c-73416711b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.574898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91efaa-b444-42fb-ad1c-73416711b357 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.575181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91efaa-b444-42fb-ad1c-73416711b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.575460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91efaa-b444-42fb-ad1c-73416711b357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.576415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91efaa-b444-42fb-ad1c-73416711b357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.578070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91efaa-b444-42fb-ad1c-73416711b357 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.589624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1175/2336] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:32:00.593962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.606306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0dc06-fa67-41f2-b4f6-f63cc002a5eb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.612028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.613032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.613429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.613708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] PATH_INFO: `/v3/projects/9151a873ebdb496ba211e1ed8a65b22e/users/3cc0892bb8864d389ceab3a51344a6cd/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.632765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9151a873ebdb496ba211e1ed8a65b22e, user_id=3cc0892bb8864d389ceab3a51344a6cd, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.635997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.653058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1162/2337] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:32:00 2022] PUT /identity/v3/projects/9151a873ebdb496ba211e1ed8a65b22e/users/3cc0892bb8864d389ceab3a51344a6cd/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:00.666919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.667218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.667505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.692462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:00.705849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] MFA Rules not processed for user `3cc0892bb8864d389ceab3a51344a6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:00.707740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:00.787883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1176/2338] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:32:00.793569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.802315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:00.889417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e7195-af6e-465d-91b9-e519c24e3d9d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:00.897756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:00.900507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:00.900960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:00.901426 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:00.902657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:00.905935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:00.941747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1163/2339] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:32:00 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:32:00.946364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:00.960260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a7e27-aaad-434e-a426-7a7562f04e6f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:01.062614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:01.063534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.063696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.063870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.064640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:01.066675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:01.095377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1177/2340] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:32:00 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:32:01.099612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:01.114363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a627d-ae78-4bcc-b51f-0b5db457a4ef admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:01.120742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:01.121481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.121740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.122023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.122853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:01.124624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:01.142711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1164/2341] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:32:01.147978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:01.163441 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155b4b-0deb-4983-919b-8e546984c0d3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:01.170220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:01.171223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.171698 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.172170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] PATH_INFO: `/v3/projects/1a76cf587a904d208591c291ed6397f1/users/09986d2e062841d89492a49397151add/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.190362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a76cf587a904d208591c291ed6397f1, user_id=09986d2e062841d89492a49397151add, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:01.193676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:01.210103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1178/2342] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:32:01 2022] PUT /identity/v3/projects/1a76cf587a904d208591c291ed6397f1/users/09986d2e062841d89492a49397151add/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:01.215602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:01.230633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5abd-8e45-4dc0-892c-12446cfd7b16 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:01.299901 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78674b9-821c-443d-af3c-1ebd1823246f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:01.300705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78674b9-821c-443d-af3c-1ebd1823246f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.301040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78674b9-821c-443d-af3c-1ebd1823246f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.301375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78674b9-821c-443d-af3c-1ebd1823246f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.302174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78674b9-821c-443d-af3c-1ebd1823246f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:01.303949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78674b9-821c-443d-af3c-1ebd1823246f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:01.315994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1165/2343] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:32:01.321989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:01.338514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327cb54-dfa9-4125-9c50-dc699ba8a81a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:01.346358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:01.347917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.348583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.349053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] PATH_INFO: `/v3/projects/1a76cf587a904d208591c291ed6397f1/users/09986d2e062841d89492a49397151add/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.353316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a76cf587a904d208591c291ed6397f1, user_id=09986d2e062841d89492a49397151add, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:01.359063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:01.372787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1179/2344] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:32:01 2022] PUT /identity/v3/projects/1a76cf587a904d208591c291ed6397f1/users/09986d2e062841d89492a49397151add/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:01.385342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.385619 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.385898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.433078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] MFA Rules not processed for user `09986d2e062841d89492a49397151add`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:01.435244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:01.514319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:01.521974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1166/2345] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:01.535675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:01.541454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:01.613280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6e2e7-d3b5-4f24-8946-75240a3fffd5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:01.618362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32c254-9fd9-488f-a464-4c5707645623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:01.619066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c254-9fd9-488f-a464-4c5707645623 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:01.619354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c254-9fd9-488f-a464-4c5707645623 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:01.619628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32c254-9fd9-488f-a464-4c5707645623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:01.626007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:01.700298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32c254-9fd9-488f-a464-4c5707645623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:01.701766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32c254-9fd9-488f-a464-4c5707645623 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:01.714150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1180/2346] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:32:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:32:04.473509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:04.567707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee513b7-0e3d-4c5d-8d85-d37636cf2eb4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:04.573270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:04.574303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:04.574696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:04.575095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] PATH_INFO: `/v3/users/3cc0892bb8864d389ceab3a51344a6cd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:04.576679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc0892bb8864d389ceab3a51344a6cd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:04.578913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:04.643390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc0892bb8864d389ceab3a51344a6cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:04.652321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc0892bb8864d389ceab3a51344a6cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:04.665463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc0892bb8864d389ceab3a51344a6cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:04.681305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc0892bb8864d389ceab3a51344a6cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:04.682077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc0892bb8864d389ceab3a51344a6cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:04.690070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1167/2347] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:32:04 2022] DELETE /identity/v3/users/3cc0892bb8864d389ceab3a51344a6cd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:04.694396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:04.694932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:04.694932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:04.715855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:04.717163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:04.809686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1181/2348] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:04.823119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:04.826014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:04.917626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30d58b-69e8-4735-baee-fe79cceb41b5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:04.923956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:04.925059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:04.925474 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:04.925880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:04.943180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:04.946708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:04.962775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1168/2349] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:32:05.572847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:05.575215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:05.604553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103c2f-6d0f-4577-ab0f-8f7e625afb1d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:05.615170 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:05.616095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:05.616398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:05.616676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] PATH_INFO: `/v3/projects/9151a873ebdb496ba211e1ed8a65b22e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:05.618544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9151a873ebdb496ba211e1ed8a65b22e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:05.620761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:05.696017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9151a873ebdb496ba211e1ed8a65b22e', 'request_id': 'req-e4d316a3-8e01-4603-af93-d86e21a861f4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.705853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9151a873ebdb496ba211e1ed8a65b22e', 'request_id': 'req-e4d316a3-8e01-4603-af93-d86e21a861f4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.715510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9151a873ebdb496ba211e1ed8a65b22e', 'request_id': 'req-e4d316a3-8e01-4603-af93-d86e21a861f4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.734634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] The token cache is being invalidate because project 9151a873ebdb496ba211e1ed8a65b22e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:32:05.734946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.735950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.738163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1182/2350] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:32:05 2022] DELETE /identity/v3/projects/9151a873ebdb496ba211e1ed8a65b22e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:05.744097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:05.755191 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd7449-dbfe-404f-8482-633779e1e70f service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:05.778977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:05.850093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:05.853038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:05.853038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:05.853038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] PATH_INFO: `/v3/users/09986d2e062841d89492a49397151add` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:05.854948 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] RBAC: Authorizing `identity:delete_user(user_id=09986d2e062841d89492a49397151add)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:05.857107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:05.938101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09986d2e062841d89492a49397151add'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.951417 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09986d2e062841d89492a49397151add'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.964657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09986d2e062841d89492a49397151add'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.993169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09986d2e062841d89492a49397151add'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:05.994372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-50b3fc36-3766-4e82-8c35-f5751d0f189a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09986d2e062841d89492a49397151add'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:06.006215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1169/2351] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:32:05 2022] DELETE /identity/v3/users/09986d2e062841d89492a49397151add => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:32:06.463764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:06.465556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:06.487431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d316a3-8e01-4603-af93-d86e21a861f4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:06.542977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:06.595888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:06.600461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:06.600918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:06.602269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] PATH_INFO: `/v3/projects/1a76cf587a904d208591c291ed6397f1` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:06.604269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a76cf587a904d208591c291ed6397f1)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:06.607272 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:06.673495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a76cf587a904d208591c291ed6397f1', 'request_id': 'req-65034712-25fa-4ce9-8ed7-d15aa5392f27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:06.682342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a76cf587a904d208591c291ed6397f1', 'request_id': 'req-65034712-25fa-4ce9-8ed7-d15aa5392f27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:06.690257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a76cf587a904d208591c291ed6397f1', 'request_id': 'req-65034712-25fa-4ce9-8ed7-d15aa5392f27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:06.712802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] The token cache is being invalidate because project 1a76cf587a904d208591c291ed6397f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:32:06.713560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:06.714787 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-65034712-25fa-4ce9-8ed7-d15aa5392f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:06.717783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1183/2352] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:32:06 2022] DELETE /identity/v3/projects/1a76cf587a904d208591c291ed6397f1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:06.725937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:06.727028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:06.727558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:06.761594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:06.763664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:06.840978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1170/2353] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:06.851914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:06.852250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:06.852505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:06.875997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:06.877446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:06.887940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1184/2354] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:06.892913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:06.904430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45638-09b9-46ea-ae51-53d358b28f7d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:06.910463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:06.911168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:06.911450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:06.911728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:06.912526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:32:06.912839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:06.915637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:06.929452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1171/2355] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:32:06.935450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:06.946033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfc018-4f4a-43a8-9ef1-cc193088b90c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:06.950995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:06.951674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:06.951959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:06.952297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:06.953013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:06.954993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:06.986173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1185/2356] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:32:06 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:32:06.990811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:07.002819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cf734-a4c6-41e0-8c50-46aab0a32c1f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:07.033033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:07.077038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:07.077879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:07.078207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:07.078524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:07.079429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:07.081591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:07.107681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1172/2357] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:32:06 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:32:07.112344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:07.124645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be806c-cfdb-4672-9b98-ddd7cadcd014 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:07.129608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:07.130282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:07.130565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:07.130839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:07.131581 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:07.134062 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:07.146808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1186/2358] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:32:07.153469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:07.165248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a7c65-8856-492d-8ef3-539fb3008074 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:07.170773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a69a8-a671-461a-9b9c-c548d60ffcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:07.171480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a69a8-a671-461a-9b9c-c548d60ffcac admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:07.171765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a69a8-a671-461a-9b9c-c548d60ffcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:07.172071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a69a8-a671-461a-9b9c-c548d60ffcac admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:07.172837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a69a8-a671-461a-9b9c-c548d60ffcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:07.174487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a69a8-a671-461a-9b9c-c548d60ffcac admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:07.186900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1173/2359] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:32:07.191542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:07.204198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0cc3e-7c26-4571-a112-72f6d20427a4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:07.209811 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:07.211373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:07.212414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:07.212743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] PATH_INFO: `/v3/projects/c7ea632f4a7340d1a7f1b11a85eb38fb/users/59f6cb436abc4d0eb0109ed13f3d19e3/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:07.232517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ea632f4a7340d1a7f1b11a85eb38fb, user_id=59f6cb436abc4d0eb0109ed13f3d19e3, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:07.235259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:07.253031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1187/2360] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:32:07 2022] PUT /identity/v3/projects/c7ea632f4a7340d1a7f1b11a85eb38fb/users/59f6cb436abc4d0eb0109ed13f3d19e3/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:32:07.266046 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:07.266553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:07.267112 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:07.305053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] MFA Rules not processed for user `59f6cb436abc4d0eb0109ed13f3d19e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:07.306343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:07.400548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1174/2361] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:07.412749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:07.415741 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:07.493614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7bec8-7f1d-4e1f-8338-46917e18a75e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:07.497765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249c772-e5ca-46ec-af10-2c4d35c257ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:07.498433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c772-e5ca-46ec-af10-2c4d35c257ec service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:07.498722 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c772-e5ca-46ec-af10-2c4d35c257ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:07.498997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c772-e5ca-46ec-af10-2c4d35c257ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:07.510486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249c772-e5ca-46ec-af10-2c4d35c257ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:07.511995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249c772-e5ca-46ec-af10-2c4d35c257ec service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:07.518188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:07.525064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1188/2362] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:32:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:32:10.871277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:11.001274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c7bf3-92d7-4b94-8b12-99857d3e3bfa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:11.009890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:11.011469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:11.011972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:11.012563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] PATH_INFO: `/v3/users/59f6cb436abc4d0eb0109ed13f3d19e3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:11.014874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f6cb436abc4d0eb0109ed13f3d19e3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:11.018928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:11.087966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:11.103117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f6cb436abc4d0eb0109ed13f3d19e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:11.112347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f6cb436abc4d0eb0109ed13f3d19e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:11.125375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f6cb436abc4d0eb0109ed13f3d19e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:11.153131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f6cb436abc4d0eb0109ed13f3d19e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:11.154318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f6cb436abc4d0eb0109ed13f3d19e3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:11.163179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1175/2363] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:32:10 2022] DELETE /identity/v3/users/59f6cb436abc4d0eb0109ed13f3d19e3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:11.168718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0efed-747d-473d-a512-311f816bf268 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:11.169587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0efed-747d-473d-a512-311f816bf268 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:11.169587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0efed-747d-473d-a512-311f816bf268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:11.189056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-31a0efed-747d-473d-a512-311f816bf268 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:11.190322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-31a0efed-747d-473d-a512-311f816bf268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:11.278653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1189/2364] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:11.290071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:11.291627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:11.377957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801adad-7b4a-45cc-bb56-1f1057d63776 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:11.382088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:11.383221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:11.383506 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:11.384017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:11.399536 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:11.402377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:11.416548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1176/2365] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:32:12.244652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0efed-747d-473d-a512-311f816bf268 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:12.247666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-31a0efed-747d-473d-a512-311f816bf268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:12.268226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:12.270679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:12.275296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0efed-747d-473d-a512-311f816bf268 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:12.297685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:12.300788 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:12.302194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44aa48-8395-4d6c-9db7-bd102b8c7d60 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:12.302531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:12.303518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] PATH_INFO: `/v3/projects/c7ea632f4a7340d1a7f1b11a85eb38fb` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:12.307460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ea632f4a7340d1a7f1b11a85eb38fb)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:12.314070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:12.314428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:12.316021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:12.316021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:12.316710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:12.331858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:12.473334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ea632f4a7340d1a7f1b11a85eb38fb', 'request_id': 'req-0d933cce-7b72-49b1-a655-72aed5190455'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:12.485691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ea632f4a7340d1a7f1b11a85eb38fb', 'request_id': 'req-0d933cce-7b72-49b1-a655-72aed5190455'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:12.505828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ea632f4a7340d1a7f1b11a85eb38fb', 'request_id': 'req-0d933cce-7b72-49b1-a655-72aed5190455'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:12.513002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:12.516621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:12.549521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] The token cache is being invalidate because project c7ea632f4a7340d1a7f1b11a85eb38fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:32:12.549521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:12.550308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0d933cce-7b72-49b1-a655-72aed5190455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:32:12.566780 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1190/2366] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:32:12 2022] DELETE /identity/v3/projects/c7ea632f4a7340d1a7f1b11a85eb38fb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:32:12.580405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1177/2367] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:32:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:32:14.905308 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9af1-375c-42bb-8d56-1438fa3d975b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:14.906164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9af1-375c-42bb-8d56-1438fa3d975b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:14.906634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9af1-375c-42bb-8d56-1438fa3d975b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:14.954356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a64e9af1-375c-42bb-8d56-1438fa3d975b None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:14.957495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a64e9af1-375c-42bb-8d56-1438fa3d975b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:15.107793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1191/2368] 10.222.0.30 () {58 vars in 967 bytes} [Thu Jun 16 18:32:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:32:15.137257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:32:15.140960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:15.260847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:15.288168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcb866-0140-4314-a7ec-1d095a16385b service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:32:15.440818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e86fa-eb71-4f03-ae75-90941ba9d831 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:32:15.445183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e86fa-eb71-4f03-ae75-90941ba9d831 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:15.445967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e86fa-eb71-4f03-ae75-90941ba9d831 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:15.446469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e86fa-eb71-4f03-ae75-90941ba9d831 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:15.460638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e86fa-eb71-4f03-ae75-90941ba9d831 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:32:15.466038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e86fa-eb71-4f03-ae75-90941ba9d831 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:32:15.485729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1178/2369] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:32:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:32:25.098639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:25.481691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=35 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:32:26.502211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a386c7-0adc-402f-84cc-1226db93616e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:32:26.502941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a386c7-0adc-402f-84cc-1226db93616e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:32:26.502941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a386c7-0adc-402f-84cc-1226db93616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:32:26.526259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a2a386c7-0adc-402f-84cc-1226db93616e None None] MFA Rules not processed for user `89f6c755dbf74640a372802f02733eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:32:26.527601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a2a386c7-0adc-402f-84cc-1226db93616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:32:26.599108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1192/2370] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:32:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:32:36.588586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=25 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:07.478204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:07.478987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:07.478987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:07.504562 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] MFA Rules not processed for user `73dc3ab6bc6448a89b26e3e1a1b5c6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:07.506542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:07.586286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1179/2371] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:33:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:33:13.587185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:13.588157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:13.588330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:13.611882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] MFA Rules not processed for user `512f01cca14b447c8f6c9eebbd828eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:13.613708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:13.712232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1193/2372] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:33:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:33:17.576420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=25 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:23.703945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=25 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:25.789640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:25.792740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:25.919149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd9ef8-2127-4b3b-bff1-06de5261c15c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:25.924544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:25.925299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:25.925585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:25.925944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:25.928353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:26.043033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:26.046187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03cd2f-541e-4a2b-97a0-3e62dc226be3 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:26.067993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1180/2373] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:33:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:33:29.244330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:29.246249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:29.332974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7699-2ece-43b1-9d5f-0e4f53ef5ffc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:29.337456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:29.338591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:29.338591 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:29.339482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:29.341344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:29.359849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:29.360707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:29.375734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1194/2374] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:33:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:33:36.059451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=54 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:39.369890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=26 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:40.735764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:40.736643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:40.736846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:40.780492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:40.783117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:40.794807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1181/2375] 10.222.0.30 () {58 vars in 967 bytes} [Thu Jun 16 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:33:40.835270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:40.837846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:40.878145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e58d17-17b0-4647-9dda-8357eefd77bc service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:40.891861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:40.893611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:40.894369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:40.894706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:40.899282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:40.909167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:40.911710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:40.922850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:40.926726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962bc78-dabf-4e89-847a-53c16b60649c service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:40.938935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea7fd-fca5-4df4-a09a-11d356b2f7fe None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:40.947628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d9ed7-b153-41c5-8456-284f5e688063 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:40.948575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d9ed7-b153-41c5-8456-284f5e688063 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:40.948987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d9ed7-b153-41c5-8456-284f5e688063 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:40.949383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d9ed7-b153-41c5-8456-284f5e688063 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:40.964409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d9ed7-b153-41c5-8456-284f5e688063 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:40.967674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d9ed7-b153-41c5-8456-284f5e688063 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:40.974409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1195/2376] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:33:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:33:40.984320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1182/2377] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:33:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:33:44.915502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f1ca-e984-4aa6-b699-c7478150dfa5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:44.916130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f1ca-e984-4aa6-b699-c7478150dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:44.916264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f1ca-e984-4aa6-b699-c7478150dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:44.942653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1f36f1ca-e984-4aa6-b699-c7478150dfa5 None None] MFA Rules not processed for user `89f6c755dbf74640a372802f02733eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:44.944111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1f36f1ca-e984-4aa6-b699-c7478150dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:44.953985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1196/2378] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:33:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:33:50.980190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=9 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:51.727965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1687bd-1e88-4777-b936-686df05bdc1a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:51.728274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1687bd-1e88-4777-b936-686df05bdc1a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:51.728550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1687bd-1e88-4777-b936-686df05bdc1a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:51.730251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1183/2379] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:33:51.767232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:51.768604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:51.769168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:51.830245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1687bd-1e88-4777-b936-686df05bdc1a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:51.832896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9f1687bd-1e88-4777-b936-686df05bdc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:51.838947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] MFA Rules not processed for user `cad094d850d946d18b683bfc04c6f3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:51.840332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:51.927313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1197/2380] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:33:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:33:51.941479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:51.943107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:51.958027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0636dce-4ce7-458d-9e09-91c69ff0c233 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:51.965307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:51.965996 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:51.966521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:51.966983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:51.979289 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:51.980983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:51.992394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1198/2381] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:33:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:33:52.001016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1687bd-1e88-4777-b936-686df05bdc1a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:52.008439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:52.009652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:52.010078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:52.010457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] PATH_INFO: `/v3/users/0d7d57c4231b49b28a606a17e42267e7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:52.011987 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7d57c4231b49b28a606a17e42267e7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:52.014397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:52.075267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7d57c4231b49b28a606a17e42267e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.083743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7d57c4231b49b28a606a17e42267e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.099390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7d57c4231b49b28a606a17e42267e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.127200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7d57c4231b49b28a606a17e42267e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.128699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7d57c4231b49b28a606a17e42267e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.142019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1184/2382] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:33:51 2022] DELETE /identity/v3/users/0d7d57c4231b49b28a606a17e42267e7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:33:52.424661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:52.457188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f24c4a-d1e3-4990-91a8-36220e9a3bb3 service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:52.579168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:52.580442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:52.580830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:52.581235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] PATH_INFO: `/v3/projects/c9fa2caa1412451aa0308c807d5c62d8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:52.582665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fa2caa1412451aa0308c807d5c62d8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:52.585580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:52.662616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fa2caa1412451aa0308c807d5c62d8', 'request_id': 'req-c238f5c0-c455-4e48-8788-0789ef72fe66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.668463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:52.674328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fa2caa1412451aa0308c807d5c62d8', 'request_id': 'req-c238f5c0-c455-4e48-8788-0789ef72fe66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.685737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fa2caa1412451aa0308c807d5c62d8', 'request_id': 'req-c238f5c0-c455-4e48-8788-0789ef72fe66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.707199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] The token cache is being invalidate because project c9fa2caa1412451aa0308c807d5c62d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:33:52.707751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.708489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.711214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1199/2383] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:33:52 2022] DELETE /identity/v3/projects/c9fa2caa1412451aa0308c807d5c62d8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:33:52.715384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:52.718423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:52.736729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc0ec3-9d2a-44df-8c23-61afcc18fe8f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:52.858885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:52.860341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:52.860817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:52.861314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] PATH_INFO: `/v3/users/eae3c2dfa052478ebc0b63fafdebd141` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:52.863660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae3c2dfa052478ebc0b63fafdebd141)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:52.869593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:52.904315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:52.944753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae3c2dfa052478ebc0b63fafdebd141'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.954393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae3c2dfa052478ebc0b63fafdebd141'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.968743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae3c2dfa052478ebc0b63fafdebd141'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.996036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae3c2dfa052478ebc0b63fafdebd141'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:52.997379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fa147313-5cb5-41b1-abe8-daeb6d8807ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae3c2dfa052478ebc0b63fafdebd141'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:53.021291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1185/2384] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:33:52 2022] DELETE /identity/v3/users/eae3c2dfa052478ebc0b63fafdebd141 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:33:53.376326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:53.378427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:53.394411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238f5c0-c455-4e48-8788-0789ef72fe66 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:53.469349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:53.470470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:53.470818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:53.471166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] PATH_INFO: `/v3/projects/36d2bd288b5c49b98df6e29aa13fa1f1` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:53.472618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d2bd288b5c49b98df6e29aa13fa1f1)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:53.474705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:53.547912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d2bd288b5c49b98df6e29aa13fa1f1', 'request_id': 'req-ad439021-58f2-4ab0-9b71-1050afdab485'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:53.560821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d2bd288b5c49b98df6e29aa13fa1f1', 'request_id': 'req-ad439021-58f2-4ab0-9b71-1050afdab485'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:53.574148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d2bd288b5c49b98df6e29aa13fa1f1', 'request_id': 'req-ad439021-58f2-4ab0-9b71-1050afdab485'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:53.596414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] The token cache is being invalidate because project 36d2bd288b5c49b98df6e29aa13fa1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:33:53.596944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:53.598115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad439021-58f2-4ab0-9b71-1050afdab485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:33:53.600998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1200/2385] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:33:53 2022] DELETE /identity/v3/projects/36d2bd288b5c49b98df6e29aa13fa1f1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:33:53.609119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:53.610555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:53.611294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:53.647639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:53.649789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:53.761784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1186/2386] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:33:53.769718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:53.770004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:53.770313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:53.791004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:53.792279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:53.801070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1201/2387] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:33:53.806176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:53.816820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502605cb-8b9f-47e2-95fe-a58a2e94cae2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:53.823011 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:53.823707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:53.823988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:53.824297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:53.825044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:33:53.825354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:53.827009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:53.841798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1187/2388] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:33:53.847418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:53.864984 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f790c7-2211-406c-a758-24fe5e586a55 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:53.873242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:53.874381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:53.874943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:53.875483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:53.876893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:53.880686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:53.923677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1202/2389] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:33:53 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:33:53.928390 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:53.940756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8c61-7182-46e2-899a-9692aece4cfe admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:54.007363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:54.008248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:54.008648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:54.009018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:54.009780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:54.011939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:54.038392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1188/2390] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:33:53 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:33:54.042719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:54.054751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb06c0d-b7a2-45d6-9be5-346f8d2d18cf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:54.059726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:54.060540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:54.060829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:54.061103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:54.061861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:54.063576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:54.076590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1203/2391] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:33:54.081434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:54.093060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a223d-5875-45b3-beba-8a83ca0dbed8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:54.098153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66903d-1895-44ba-9379-56856a80807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:54.098931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66903d-1895-44ba-9379-56856a80807a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:54.099257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66903d-1895-44ba-9379-56856a80807a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:54.099658 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66903d-1895-44ba-9379-56856a80807a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:54.100563 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66903d-1895-44ba-9379-56856a80807a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:54.102260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66903d-1895-44ba-9379-56856a80807a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:54.114646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1189/2392] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:33:54.119329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:54.130821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaa7b8-e6d3-4635-afd4-dea01a561a3f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:54.136570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:54.137518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:54.137801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:54.138075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] PATH_INFO: `/v3/projects/4d9f254f568e4bd98b2e4087c3a32c04/users/0cc31f4b39704173858410ae52ac2c66/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:54.157123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9f254f568e4bd98b2e4087c3a32c04, user_id=0cc31f4b39704173858410ae52ac2c66, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:54.159883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:54.175451 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1204/2393] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:33:54 2022] PUT /identity/v3/projects/4d9f254f568e4bd98b2e4087c3a32c04/users/0cc31f4b39704173858410ae52ac2c66/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:33:54.181582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:54.181902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:54.182187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:54.189699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:54.203443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:33:54.205581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:54.277111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1190/2394] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:33:54.288959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:54.290951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:54.304006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:33:54.384614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773dfdd-eeb8-436f-a46d-26138e0aa249 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:54.389058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:54.389730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:54.390018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:54.390295 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:54.401458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:54.402969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:54.415792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1205/2395] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:33:58.952746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:33:58.955120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:59.039175 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953f29-02cb-4f05-9eed-5c79518c5eff None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:33:59.047249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:33:59.050480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:33:59.051291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:33:59.051984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:33:59.058770 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:33:59.239069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:33:59.242295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d933a-f3a2-4319-bd21-809d97a5aa8d service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:33:59.266483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1191/2396] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:33:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:01.253254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:01.255607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:01.416689 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e585e0-95d7-4251-b449-d09cb8d08ddf service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:01.445238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:01.446488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:01.446986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:01.447468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:01.451599 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:01.557359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:01.559208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:01.572104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1206/2397] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:01.855230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:01.856167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:01.856167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:01.917032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] MFA Rules not processed for user `0cc31f4b39704173858410ae52ac2c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:01.918355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:02.038512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:02.071094 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1192/2398] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:02.072523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:02.090943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc8fc6-3b7a-4669-be4f-615efa8f572a service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:02.097096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:02.097841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:02.098123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:02.098394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:02.101498 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:02.103682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:02.146269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1207/2399] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:34:02 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:34:02.151473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:02.182210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5c40-6561-475a-836e-57d0a9ea74f4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:02.299806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:02.300774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:02.301145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:02.301633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:02.303545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:02.306785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:02.332757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1193/2400] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:34:02 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:34:02.338060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:02.351098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0529b8-822d-4143-8e4b-f9eb40e0b983 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:02.356204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:02.356923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:02.357218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:02.357491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:02.358310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:02.359981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:02.392928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1208/2401] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:34:02.394095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:02.405812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38c56-10dc-475e-a663-20f779a2f632 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:02.413638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:02.414587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:02.415059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:02.415247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] PATH_INFO: `/v3/projects/f955e3c1c6ed4b99b5ab4009d470a806/users/2deb86b8eef34c2d934d63f5ead001b4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:02.434874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f955e3c1c6ed4b99b5ab4009d470a806, user_id=2deb86b8eef34c2d934d63f5ead001b4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:02.438693 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:02.469155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1194/2402] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:34:02 2022] PUT /identity/v3/projects/f955e3c1c6ed4b99b5ab4009d470a806/users/2deb86b8eef34c2d934d63f5ead001b4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:34:02.470227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:02.489241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cdcd8-80af-4c52-9e6e-8ec95416699a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:02.502127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:02.588353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4384201-9bc4-401b-bc49-9d58661be353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:02.589148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4384201-9bc4-401b-bc49-9d58661be353 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:02.589501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4384201-9bc4-401b-bc49-9d58661be353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:02.589839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4384201-9bc4-401b-bc49-9d58661be353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:02.590665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4384201-9bc4-401b-bc49-9d58661be353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:02.592662 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4384201-9bc4-401b-bc49-9d58661be353 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:02.605026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1209/2403] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:34:02.610009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:02.624433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235d4ff-00ba-4d00-b0b1-c2dbeea81bee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:02.629817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:02.630784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:02.631795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:02.633044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] PATH_INFO: `/v3/projects/f955e3c1c6ed4b99b5ab4009d470a806/users/2deb86b8eef34c2d934d63f5ead001b4/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:02.635387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f955e3c1c6ed4b99b5ab4009d470a806, user_id=2deb86b8eef34c2d934d63f5ead001b4, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:02.638176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:02.652664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1195/2404] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:34:02 2022] PUT /identity/v3/projects/f955e3c1c6ed4b99b5ab4009d470a806/users/2deb86b8eef34c2d934d63f5ead001b4/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:34:11.289926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddda439-dcf5-445e-a097-92b60be5335f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:11.290666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddda439-dcf5-445e-a097-92b60be5335f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:11.290965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddda439-dcf5-445e-a097-92b60be5335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:11.343660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1ddda439-dcf5-445e-a097-92b60be5335f None None] MFA Rules not processed for user `2deb86b8eef34c2d934d63f5ead001b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:11.345900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ddda439-dcf5-445e-a097-92b60be5335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:11.468245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1210/2405] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:11.485439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:11.488279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:11.601320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ef3f1-c544-4a14-9525-9519d041e66b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:11.607445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:11.608411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:11.608703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:11.608986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:11.678220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:11.715533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:11.719410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:11.742028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1196/2406] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:13.510138 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:13.511013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:13.511013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:13.543116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] MFA Rules not processed for user `0cc31f4b39704173858410ae52ac2c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:13.545383 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:13.553936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1211/2407] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:34:16.126860 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:16.234056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72503946-fb3f-4965-9830-4913e50c94a5 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:16.239835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:16.241416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:16.241805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:16.242181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:16.244560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:16.278126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:16.285262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:16.303687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1197/2408] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:21.082362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:21.118861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb48454-86b9-4200-af54-7ee83350b73f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:21.137190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:21.138686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:21.139157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:21.139630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:21.141993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:21.158410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:21.163560 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f175dcd-bb87-498a-916b-a344577ddddf service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:21.181055 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1212/2409] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:22.114627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:22.117416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:22.137454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb74b7-419e-42a1-8ba5-5ce1d095bd24 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:22.143834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:22.145016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:22.145531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:22.146816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:22.151028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:22.257340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:22.259800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:22.282103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1198/2410] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:23.486648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:23.487253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:23.487253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:23.517103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:23.518912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:23.611775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1213/2411] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:34:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:23.631182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:23.633545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:23.748950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c4392-9293-47f2-aa3d-be6301e22e27 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:23.763814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:23.764586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:23.764880 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:23.765328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:23.776361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:23.778054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:23.789487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1199/2412] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:34:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:34:28.179905 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:28.186938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:28.238061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5aef4-1b23-405f-b7c2-4b46ec42f2a4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:28.245814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:28.246918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:28.247356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:28.247563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] PATH_INFO: `/v3/users/512f01cca14b447c8f6c9eebbd828eb2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:28.249142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512f01cca14b447c8f6c9eebbd828eb2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:28.251544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:28.273026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:28.339413 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512f01cca14b447c8f6c9eebbd828eb2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.363514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512f01cca14b447c8f6c9eebbd828eb2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.374656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512f01cca14b447c8f6c9eebbd828eb2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.375525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512f01cca14b447c8f6c9eebbd828eb2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.376220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19dce718-728b-4ba8-a488-579a49030cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512f01cca14b447c8f6c9eebbd828eb2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.397054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1214/2413] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:34:28 2022] DELETE /identity/v3/users/512f01cca14b447c8f6c9eebbd828eb2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:34:28.713450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:28.739586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:28.750279 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bcca-6326-40c0-9f3e-cda1a1da55ef service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:28.841340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:28.842823 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:28.843086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:28.843398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] PATH_INFO: `/v3/projects/b70bbccf07e3404cb94ba59201061a6a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:28.844828 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70bbccf07e3404cb94ba59201061a6a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:28.846873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:28.926136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70bbccf07e3404cb94ba59201061a6a', 'request_id': 'req-02a2f31b-0570-497b-9606-4b62db038591'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.933475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70bbccf07e3404cb94ba59201061a6a', 'request_id': 'req-02a2f31b-0570-497b-9606-4b62db038591'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.941866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70bbccf07e3404cb94ba59201061a6a', 'request_id': 'req-02a2f31b-0570-497b-9606-4b62db038591'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.961820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] The token cache is being invalidate because project b70bbccf07e3404cb94ba59201061a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:34:28.962121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.962804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-02a2f31b-0570-497b-9606-4b62db038591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:34:28.964751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1200/2414] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:34:28 2022] DELETE /identity/v3/projects/b70bbccf07e3404cb94ba59201061a6a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:34:28.971893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:28.972411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:28.972796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:28.995892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:28.997205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:29.102742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1215/2415] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:29.105243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.105796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.106306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.138856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:29.140281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:29.152014 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1201/2416] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:29.160148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.179856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adc022-e66b-488c-8d20-a8e68c64f6ef None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:29.185821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:29.186702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.186877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.187248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.188028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:34:29.188511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:29.190199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:29.207069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1216/2417] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:34:29.214810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.229362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba53792-1626-4061-a29e-3b6cbc27ff5d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:29.237177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:29.238449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.238958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.239466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.241580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:29.246168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:29.320804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1202/2418] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:34:29 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:34:29.326172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.348300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb6def-16aa-4cb9-896b-742c25dff2c3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:29.462782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:29.463819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.464494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.464938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.466546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:29.468829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:29.515537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1217/2419] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:34:29 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:34:29.519957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.543937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680fced-3d7c-4e99-9b55-7361110c81d9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:29.550849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:29.556696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.556978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.557308 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.558433 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:29.561122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:29.584963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1203/2420] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:34:29.592217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.613345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106a2f0-8a7e-4479-a3cd-7a0d394d9527 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:29.625185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817011b-9ac6-4356-90a9-7df38739271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:29.626025 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817011b-9ac6-4356-90a9-7df38739271e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.626320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817011b-9ac6-4356-90a9-7df38739271e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.626594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817011b-9ac6-4356-90a9-7df38739271e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.627347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817011b-9ac6-4356-90a9-7df38739271e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:29.629086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817011b-9ac6-4356-90a9-7df38739271e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:29.632832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:29.649687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1218/2421] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:34:29.653486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.671591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512ccb-913c-4cba-8287-ef417cd0a780 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:29.685362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:29.686737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.687100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.687500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] PATH_INFO: `/v3/projects/7a8e1d26bdfc400ebd1e070b09db615e/users/ebdf8e3cd1b44dcba37cdcfb4af7db2b/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.736016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8e1d26bdfc400ebd1e070b09db615e, user_id=ebdf8e3cd1b44dcba37cdcfb4af7db2b, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:29.741785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:29.784010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1204/2422] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:34:29 2022] PUT /identity/v3/projects/7a8e1d26bdfc400ebd1e070b09db615e/users/ebdf8e3cd1b44dcba37cdcfb4af7db2b/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:34:29.790455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:29.790738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:29.791017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:29.830701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:29.832285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:29.980872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1219/2423] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:29.992003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:29.998169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:30.059296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:30.152250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcd5c5-9c60-45e8-8b64-3f2c071171c0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:30.162381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:30.164118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:30.164617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:30.165100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:30.184705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:30.189849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:30.210673 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1205/2424] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:30.453094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:30.531069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dce7d2-8afd-4653-9c43-96d990f71fc0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:30.549679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:30.550728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:30.551187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:30.551623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:30.556108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:30.568912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:30.570830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733c1e6-f11c-44c9-a3c4-57e2914e9335 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:30.583347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1220/2425] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:31.662867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:31.665238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:31.820742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b4769-d9e4-4f16-9aec-2f39c2f054d1 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:31.837954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:31.840547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:31.841643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:31.842493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:31.848591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:31.927691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:31.933043 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:31.950797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1206/2426] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:40.578377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=34 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:41.940282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=48 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:34:42.016513 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1ac9-129d-4c9e-a3bd-4de9b80f7451 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:42.018316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1ac9-129d-4c9e-a3bd-4de9b80f7451 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:42.018859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1ac9-129d-4c9e-a3bd-4de9b80f7451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:42.068793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f29a1ac9-129d-4c9e-a3bd-4de9b80f7451 None None] MFA Rules not processed for user `ebdf8e3cd1b44dcba37cdcfb4af7db2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:42.070569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f29a1ac9-129d-4c9e-a3bd-4de9b80f7451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:42.228287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1221/2427] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:42.244220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:42.245744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:42.362668 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954abc5-1156-4ec2-af36-cea717cc4c83 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:42.369380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:42.370072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:42.370352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:42.370624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:42.384424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:42.386311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:42.401890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1207/2428] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:45.950024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f5a3a-cdf7-464b-8904-6fe8c88e5cb8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:45.952792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f5a3a-cdf7-464b-8904-6fe8c88e5cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:45.953909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f5a3a-cdf7-464b-8904-6fe8c88e5cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:45.993453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3b4f5a3a-cdf7-464b-8904-6fe8c88e5cb8 None None] MFA Rules not processed for user `ebdf8e3cd1b44dcba37cdcfb4af7db2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:45.996250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3b4f5a3a-cdf7-464b-8904-6fe8c88e5cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:46.024594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1222/2429] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:34:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:47.327313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:47.370159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b60349-5692-48ad-9bf3-a71348b1ae70 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:47.388587 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:47.390877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:47.391601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:47.392316 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:47.397835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:47.436674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:47.440626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:47.470565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1208/2430] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:52.618774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:52.619539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:52.619875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:52.652523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:34:52.654736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:52.767466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1223/2431] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:34:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:34:52.781198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:52.783404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:52.906885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af040d5-0d5a-4fba-b867-2f00dc2c758f service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:52.915207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:52.916336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:52.916849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:52.917313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:52.933149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:52.936783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:52.957077 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1209/2432] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:34:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:34:55.477778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:55.509104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b917f-b0ee-4f04-aea0-009fbfd32983 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:34:55.516266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:34:55.517835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:34:55.519592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:34:55.520514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:34:55.524421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:34:55.684559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:34:55.686514 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf053a-4ea4-4b3d-961a-bf322a902cb7 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:34:55.701354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1224/2433] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:34:59.919997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:34:59.991903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a4fc9-968e-4c24-8181-db61bf72ad78 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:00.026030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:00.027168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:00.027937 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:00.028404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:00.031861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:00.071695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:00.079805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:00.145342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1210/2434] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:34:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:35:05.695052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:10.124086 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=78 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:10.145322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:10.146165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:10.146165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:10.190858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:10.196478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:10.210317 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1225/2435] 10.222.0.30 () {58 vars in 967 bytes} [Thu Jun 16 18:35:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:10.235544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:10.263564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4be7-93c8-4d3d-b857-ccd666c020a1 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:10.293049 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:10.294232 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:10.295022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:10.295920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:10.314920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:10.318700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:10.337897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1211/2436] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:35:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:35:13.032575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:13.034298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:13.048508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51ea70-39bc-4b02-bbc8-65bb8a23e9d5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:13.053785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:13.054519 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:13.054847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:13.055162 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:13.057219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:13.072807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:13.074335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:13.086563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1226/2437] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:19.829003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:19.832550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:19.859060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc76b2-72bc-4c2c-a818-a3c7be34ee6c service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:19.865286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:19.866492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:19.866819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:19.867134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] PATH_INFO: `/v3/users/89f6c755dbf74640a372802f02733eb3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:19.868627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f6c755dbf74640a372802f02733eb3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:19.870736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:19.933565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f6c755dbf74640a372802f02733eb3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:19.943944 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f6c755dbf74640a372802f02733eb3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:19.959458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f6c755dbf74640a372802f02733eb3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:19.978527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f6c755dbf74640a372802f02733eb3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:19.979319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b40149d9-23f5-46ae-8f5c-e7835d0709d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f6c755dbf74640a372802f02733eb3'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:19.986759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1212/2438] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:19 2022] DELETE /identity/v3/users/89f6c755dbf74640a372802f02733eb3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:20.266675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:20.290109 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed9332-a2b4-476a-bc59-52a5e868fc4d service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:20.373012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:20.374458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.374964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.375412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] PATH_INFO: `/v3/projects/1f0473c6214a4428be438fabc530012b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.377936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0473c6214a4428be438fabc530012b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:20.380666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:20.461877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0473c6214a4428be438fabc530012b', 'request_id': 'req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:20.475171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0473c6214a4428be438fabc530012b', 'request_id': 'req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:20.486511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0473c6214a4428be438fabc530012b', 'request_id': 'req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:20.513182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] The token cache is being invalidate because project 1f0473c6214a4428be438fabc530012b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:20.513701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:20.514873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6ba732c9-f3f0-4a9a-a5ab-4e038eb15df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:20.517929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1227/2439] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:20 2022] DELETE /identity/v3/projects/1f0473c6214a4428be438fabc530012b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:20.524132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.524423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.524702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.546677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:20.548387 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:20.624959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1213/2440] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:20.636280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.636784 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.637293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.673324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:20.674576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:20.684224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1228/2441] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:20.688817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:20.700396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89073a27-eb66-4e08-9463-65adaf2e1d5e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:20.705675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:20.706377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.706664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.706942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.707691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:35:20.708547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:20.710659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:20.725866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1214/2442] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:35:20.732310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:20.742531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e6b6a-409f-41d2-aa50-2d1ee037c682 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:20.747499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:20.748220 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.748594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.748813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.749561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:20.751557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:20.792929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1229/2443] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:20 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:20.798311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:20.817105 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6baf7-9d87-4bef-b2c7-41f4e766689d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:20.883130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:20.906565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:20.907438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.907724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.908002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.908764 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:20.910741 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:20.940075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1215/2444] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:20 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:20.945880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:20.958860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468ff89-5733-40de-b39b-25b1a6a3324b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:20.964306 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:20.965014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:20.965285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:20.965562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:20.966306 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:20.967942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:20.983161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1230/2445] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:35:20.987514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:21.001174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c0a7-7b92-4005-9df1-1e9c37531b9c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:21.008036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011bdad-575d-48f0-b098-abc9f6ed6461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:21.008827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011bdad-575d-48f0-b098-abc9f6ed6461 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:21.009141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011bdad-575d-48f0-b098-abc9f6ed6461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:21.009481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011bdad-575d-48f0-b098-abc9f6ed6461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:21.010326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011bdad-575d-48f0-b098-abc9f6ed6461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:21.012143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011bdad-575d-48f0-b098-abc9f6ed6461 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:21.024986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1216/2446] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:35:21.029955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:21.041785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b84803-2c9e-46b0-960f-6a5775509425 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:21.049098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:21.050019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:21.050305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:21.050585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] PATH_INFO: `/v3/projects/9912e6b0e37344ff9c319165892bd814/users/de673266a65c478c8700a46d6e842413/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:21.069544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9912e6b0e37344ff9c319165892bd814, user_id=de673266a65c478c8700a46d6e842413, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:21.072307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:21.089833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1231/2447] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:21 2022] PUT /identity/v3/projects/9912e6b0e37344ff9c319165892bd814/users/de673266a65c478c8700a46d6e842413/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:21.101364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:21.101902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:21.102277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:21.136101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] MFA Rules not processed for user `de673266a65c478c8700a46d6e842413`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:21.137371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:21.246471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1217/2448] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:21.264333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:21.266396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:21.278579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:21.394992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9010f3-5fb8-4946-bc6d-f51140629636 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:21.401247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e61628-27a1-4e85-99a7-b24a9e2bf313 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:21.401962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61628-27a1-4e85-99a7-b24a9e2bf313 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:21.402238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61628-27a1-4e85-99a7-b24a9e2bf313 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:21.402508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61628-27a1-4e85-99a7-b24a9e2bf313 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:21.413056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e61628-27a1-4e85-99a7-b24a9e2bf313 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:21.414726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e61628-27a1-4e85-99a7-b24a9e2bf313 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:21.426177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1232/2449] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:29.199303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:29.287826 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b77ac-78e3-4d62-acaa-42a2fbbb75e9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:29.294679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:29.296041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:29.296041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:29.296396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] PATH_INFO: `/v3/users/de673266a65c478c8700a46d6e842413` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:29.297976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de673266a65c478c8700a46d6e842413)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:29.299975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:29.366700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de673266a65c478c8700a46d6e842413'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:29.376861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de673266a65c478c8700a46d6e842413'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:29.379813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:29.390026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de673266a65c478c8700a46d6e842413'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:29.408145 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de673266a65c478c8700a46d6e842413'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:29.408932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de673266a65c478c8700a46d6e842413'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:29.417872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1218/2450] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:29 2022] DELETE /identity/v3/users/de673266a65c478c8700a46d6e842413 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:29.423912 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:29.424767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:29.424767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:29.446058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:29.447329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:29.531232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1233/2451] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:29.543652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:29.545586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:29.641127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166ff7-76e7-4c59-99a4-d262bdb48845 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:29.647931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8623e-2316-4641-9fb1-664200966d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:29.648672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8623e-2316-4641-9fb1-664200966d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:29.648956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8623e-2316-4641-9fb1-664200966d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:29.649236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8623e-2316-4641-9fb1-664200966d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:29.663781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8623e-2316-4641-9fb1-664200966d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:29.665298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8623e-2316-4641-9fb1-664200966d57 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:29.679362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1219/2452] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:30.079307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.081027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:30.101936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba8fe-884b-4393-b1ce-cd5ca4e4ab37 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.107828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.108896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.109076 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.109321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] PATH_INFO: `/v3/projects/9912e6b0e37344ff9c319165892bd814` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.110735 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9912e6b0e37344ff9c319165892bd814)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.112823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.180217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9912e6b0e37344ff9c319165892bd814', 'request_id': 'req-be060959-c4a8-4d82-ae55-ede42adf607b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:30.187495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9912e6b0e37344ff9c319165892bd814', 'request_id': 'req-be060959-c4a8-4d82-ae55-ede42adf607b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:30.195459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9912e6b0e37344ff9c319165892bd814', 'request_id': 'req-be060959-c4a8-4d82-ae55-ede42adf607b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:30.212353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] The token cache is being invalidate because project 9912e6b0e37344ff9c319165892bd814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:30.212643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:30.213336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-be060959-c4a8-4d82-ae55-ede42adf607b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:30.215064 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1234/2453] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:30 2022] DELETE /identity/v3/projects/9912e6b0e37344ff9c319165892bd814 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:30.222404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.223063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.223063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.249555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:30.251085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:30.346187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1220/2454] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:30.355436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.355720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.355960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.385770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:30.387916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:30.402255 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1235/2455] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:30.406957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.419908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f3c8-eebd-4f82-adc9-eec667b93c88 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.425211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.425902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.426188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.427363 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.428440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:35:30.428757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.430461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.446087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1221/2456] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:35:30.451014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.464278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de212f2-f193-46a0-86a9-35a45372fa99 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.470227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.471282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.471650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.471973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.472792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.475620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.480692 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:30.519448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1236/2457] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:30 2022] POST /identity/v3/projects => generated 403 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:35:30.522682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.540682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68293d8-4290-4ac5-96e6-4f0181a7815a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.615006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.616225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.616561 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.616976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.617760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.619793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.631274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:30.647829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1222/2458] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:30 2022] POST /identity/v3/users => generated 302 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:30.652397 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.664962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b916e4-7fd5-4082-9274-c03a118c680b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.669995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.670691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.670975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.671492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.673023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.674686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.690824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1237/2459] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:30.698151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.716831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a453df6-70c9-46ae-92e5-cead63314d9c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.725464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb52dd2-a2f7-46cd-aeaf-4663887e0ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.726739 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb52dd2-a2f7-46cd-aeaf-4663887e0ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.727278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb52dd2-a2f7-46cd-aeaf-4663887e0ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.728201 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb52dd2-a2f7-46cd-aeaf-4663887e0ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.729614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb52dd2-a2f7-46cd-aeaf-4663887e0ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.732581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb52dd2-a2f7-46cd-aeaf-4663887e0ca5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.754214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1223/2460] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:30.760718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.772673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0440-4be9-447a-9934-1316df9112a0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:30.778116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:30.779039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.779324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.779604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] PATH_INFO: `/v3/projects/4cd7177c023b4a5292256fa24699fb6e/users/9b558d88ea3a4f1eb3e951d111734690/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.808135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd7177c023b4a5292256fa24699fb6e, user_id=9b558d88ea3a4f1eb3e951d111734690, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:30.813261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:30.834676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1238/2461] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:30 2022] PUT /identity/v3/projects/4cd7177c023b4a5292256fa24699fb6e/users/9b558d88ea3a4f1eb3e951d111734690/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:30.840493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f222223-6e0c-4ad7-a1f8-d02312190cf4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:30.840773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f222223-6e0c-4ad7-a1f8-d02312190cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:30.841037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f222223-6e0c-4ad7-a1f8-d02312190cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:30.860909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3f222223-6e0c-4ad7-a1f8-d02312190cf4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:30.862150 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f222223-6e0c-4ad7-a1f8-d02312190cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:30.937518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1224/2462] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:30.949508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:30.952606 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:31.043974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce258f6-a4d4-4249-9f15-7bc464d21643 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:31.050919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:31.052923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:31.053099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:31.053574 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:31.071875 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:31.075060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:31.093609 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1239/2463] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:31.957662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:31.959013 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:31.959773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.009088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] MFA Rules not processed for user `9b558d88ea3a4f1eb3e951d111734690`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:32.011084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:32.123858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1225/2464] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:32.130941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:32.144186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34e61e-2d11-4ffd-a549-15e2d28abf7a service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:32.152532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:32.154043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:32.154043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:32.154431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.155589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:32.159896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:32.207748 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1240/2465] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:32 2022] POST /identity/v3/projects => generated 403 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:35:32.210369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:32.223523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164838d-c8ab-4036-87d7-6d47038b9519 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:32.307466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:32.308234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:32.308514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:32.308792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.309505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:32.311554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:32.337033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1226/2466] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:32 2022] POST /identity/v3/users => generated 308 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:32.343274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:32.365364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa85b12-644e-42fb-b35f-b95fc7c488dc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:32.373733 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:32.375122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:32.375531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:32.376250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.377511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:32.380216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:32.399518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1241/2467] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:32.404556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:32.414866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:32.429095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b361e4-85ac-41fe-bd1e-4f989728885e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:32.437711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:32.439645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:32.440525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:32.441213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] PATH_INFO: `/v3/projects/bd1fd4d15b33479f895418b74e9d9b64/users/7e930ed3a4514f5b8660ae2aa6b8df95/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.469356 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1fd4d15b33479f895418b74e9d9b64, user_id=7e930ed3a4514f5b8660ae2aa6b8df95, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:32.473001 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:32.492187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1227/2468] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:32 2022] PUT /identity/v3/projects/bd1fd4d15b33479f895418b74e9d9b64/users/7e930ed3a4514f5b8660ae2aa6b8df95/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:32.497432 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:32.519746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5874d40-bfdc-436f-aaff-04743710f717 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:32.596639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66946c-efa8-4f91-8faf-d4b5f35d7065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:32.597358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66946c-efa8-4f91-8faf-d4b5f35d7065 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:32.597595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66946c-efa8-4f91-8faf-d4b5f35d7065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:32.597868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66946c-efa8-4f91-8faf-d4b5f35d7065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.599792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66946c-efa8-4f91-8faf-d4b5f35d7065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:32.601124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66946c-efa8-4f91-8faf-d4b5f35d7065 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:32.612876 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1242/2469] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:32.618212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:32.630781 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfa450-3186-4498-a29e-17b5f53f8b5e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:32.638187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:32.639107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:32.639394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:32.639699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] PATH_INFO: `/v3/projects/bd1fd4d15b33479f895418b74e9d9b64/users/7e930ed3a4514f5b8660ae2aa6b8df95/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:32.641869 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1fd4d15b33479f895418b74e9d9b64, user_id=7e930ed3a4514f5b8660ae2aa6b8df95, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:32.644656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:32.657780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1228/2470] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:32 2022] PUT /identity/v3/projects/bd1fd4d15b33479f895418b74e9d9b64/users/7e930ed3a4514f5b8660ae2aa6b8df95/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:33.671142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:33.671799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:33.671799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:33.727297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] MFA Rules not processed for user `7e930ed3a4514f5b8660ae2aa6b8df95`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:33.729052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:33.736423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:33.846001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1243/2471] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:34.452234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:34.455716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:34.579749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319b75-0e33-4cfe-8d9e-5b41e549e915 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:34.591792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:34.592688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:34.593219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:34.593569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:34.596266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:34.612652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:34.615898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:34.635831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1229/2472] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:35.270657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:35.375495 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c3227-9ddf-4fed-b9bb-69b63e9a4d19 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:35.382037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:35.382939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:35.383233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:35.383505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] PATH_INFO: `/v3/users/9b558d88ea3a4f1eb3e951d111734690` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:35.384825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b558d88ea3a4f1eb3e951d111734690)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:35.386783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:35.472682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b558d88ea3a4f1eb3e951d111734690'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.490065 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b558d88ea3a4f1eb3e951d111734690'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.499303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b558d88ea3a4f1eb3e951d111734690'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.505248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b558d88ea3a4f1eb3e951d111734690'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.506711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b558d88ea3a4f1eb3e951d111734690'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.530954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1244/2473] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:35 2022] DELETE /identity/v3/users/9b558d88ea3a4f1eb3e951d111734690 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:35.774391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:35.793633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7778c-cfe3-4113-85f4-b056518b244d service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:35.883121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:35.884638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:35.885012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:35.885799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] PATH_INFO: `/v3/projects/4cd7177c023b4a5292256fa24699fb6e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:35.887648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd7177c023b4a5292256fa24699fb6e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:35.891489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:35.975595 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd7177c023b4a5292256fa24699fb6e', 'request_id': 'req-6836681e-b855-43d3-85de-7914d82da2c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.982737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:35.985033 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd7177c023b4a5292256fa24699fb6e', 'request_id': 'req-6836681e-b855-43d3-85de-7914d82da2c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:35.999578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd7177c023b4a5292256fa24699fb6e', 'request_id': 'req-6836681e-b855-43d3-85de-7914d82da2c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.020870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] The token cache is being invalidate because project 4cd7177c023b4a5292256fa24699fb6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:36.021222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.021999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.023801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1230/2474] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:35 2022] DELETE /identity/v3/projects/4cd7177c023b4a5292256fa24699fb6e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:36.028359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:36.030226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:36.048659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2aac3-e41e-40bd-8bfd-a8f31bb945b9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:36.072693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:36.126129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:36.127659 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:36.128177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:36.128677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] PATH_INFO: `/v3/users/7e930ed3a4514f5b8660ae2aa6b8df95` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:36.131486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e930ed3a4514f5b8660ae2aa6b8df95)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:36.135192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:36.206882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e930ed3a4514f5b8660ae2aa6b8df95'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.227595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e930ed3a4514f5b8660ae2aa6b8df95'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.239480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e930ed3a4514f5b8660ae2aa6b8df95'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.241049 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e930ed3a4514f5b8660ae2aa6b8df95'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.241919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8f2fe52e-b8ae-4ced-b629-07ded08dd5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e930ed3a4514f5b8660ae2aa6b8df95'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.262191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1245/2475] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:36 2022] DELETE /identity/v3/users/7e930ed3a4514f5b8660ae2aa6b8df95 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:36.528684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:36.530726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:36.546500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836681e-b855-43d3-85de-7914d82da2c6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:36.632494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:36.634024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:36.634516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:36.635036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] PATH_INFO: `/v3/projects/bd1fd4d15b33479f895418b74e9d9b64` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:36.637314 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1fd4d15b33479f895418b74e9d9b64)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:36.640171 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:36.713385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1fd4d15b33479f895418b74e9d9b64', 'request_id': 'req-b6abf310-8bdf-40d2-a443-e1872a4c4480'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.721717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1fd4d15b33479f895418b74e9d9b64', 'request_id': 'req-b6abf310-8bdf-40d2-a443-e1872a4c4480'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.730701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1fd4d15b33479f895418b74e9d9b64', 'request_id': 'req-b6abf310-8bdf-40d2-a443-e1872a4c4480'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.748003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] The token cache is being invalidate because project bd1fd4d15b33479f895418b74e9d9b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:36.748487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.749452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b6abf310-8bdf-40d2-a443-e1872a4c4480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:36.751257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1231/2476] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:36 2022] DELETE /identity/v3/projects/bd1fd4d15b33479f895418b74e9d9b64 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:36.758270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:36.759160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:36.759541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:36.780910 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:36.782299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:36.884314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1246/2477] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:36.896492 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:36.897156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:36.897156 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:36.919441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:36.921116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:36.933420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1232/2478] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:36.938435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:36.952168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78614ef7-510b-47cb-bdb2-e4db640c7dbf None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:36.958353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:36.959122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:36.959589 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:36.959871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:36.961111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:35:36.961426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:36.963124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:36.978664 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1247/2479] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:35:36.985841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:36.998000 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572f8bf-3a8c-49cc-958f-cb4c9affde07 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.002876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.003569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.003855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.004160 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.005797 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.007852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.043018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1233/2480] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:36 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:35:37.047732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.060540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecd83b-8667-4e86-8ed7-a91894587fe4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.097310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:37.154861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.155789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.156489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.156904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.157839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.161354 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.189801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1248/2481] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:37 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:37.194556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.206792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c206f-f2bc-45d1-ae8a-a00e924bf05d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.214369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.215217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.215609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.216007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.218211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.219898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.240088 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1234/2482] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:37.246097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.259348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0778f-1ceb-4ec4-888e-110fdb9f7cc8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.264881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736672b3-825a-4b6c-913c-1f2cb44dcdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.265594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736672b3-825a-4b6c-913c-1f2cb44dcdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.265882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736672b3-825a-4b6c-913c-1f2cb44dcdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.266175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736672b3-825a-4b6c-913c-1f2cb44dcdd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.266940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736672b3-825a-4b6c-913c-1f2cb44dcdd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.268696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736672b3-825a-4b6c-913c-1f2cb44dcdd9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.282406 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1249/2483] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:37.289496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.311687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db871a-9afe-4301-827a-9a66b50d808e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.320428 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.322065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.322569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.323151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] PATH_INFO: `/v3/projects/ea9a4d13be1344a2a70cd043f142ae55/users/19482cf9fba54af78d62e6276ce0bf41/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.353265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a4d13be1344a2a70cd043f142ae55, user_id=19482cf9fba54af78d62e6276ce0bf41, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.358294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.378675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1235/2484] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:37 2022] PUT /identity/v3/projects/ea9a4d13be1344a2a70cd043f142ae55/users/19482cf9fba54af78d62e6276ce0bf41/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:37.392904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.393344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.393723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.436628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] MFA Rules not processed for user `19482cf9fba54af78d62e6276ce0bf41`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:37.438043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:37.568896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1250/2485] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:37.574402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.601239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:37.700302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b1bd7-ca18-4936-8117-b2f10b0a64ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.705413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.706151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.706500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.706760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.707672 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.710911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.747633 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1236/2486] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:37 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:37.753407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.768442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee132-2092-40eb-ad48-a6b40ecd3c61 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.849156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.849668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.850069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.850447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.852239 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.855629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.884759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1251/2487] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:37 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:37.890418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.911513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b24ae-f042-45df-a4d2-1ff092078da3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.920918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.921752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.922144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.922551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.923959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.925799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:37.940012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1237/2488] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:37.945225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:37.958090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9afc3f-8bde-49be-bd7d-f9a70fe57499 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:37.965209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:37.966198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:37.966527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:37.966991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] PATH_INFO: `/v3/projects/7049f613aa064f0b894aa66fff3f4f76/users/9f683c782d984b3fb36d39b13f9e3d9d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:37.986757 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7049f613aa064f0b894aa66fff3f4f76, user_id=9f683c782d984b3fb36d39b13f9e3d9d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:37.990281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:38.006853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1252/2489] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:37 2022] PUT /identity/v3/projects/7049f613aa064f0b894aa66fff3f4f76/users/9f683c782d984b3fb36d39b13f9e3d9d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:38.013135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:38.027359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526e9b-62d6-422f-871f-1a80037a4c91 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:38.106093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c0b3d-e6b8-45ed-b9ec-0de0276b6675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:38.106804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c0b3d-e6b8-45ed-b9ec-0de0276b6675 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.107164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c0b3d-e6b8-45ed-b9ec-0de0276b6675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.107526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c0b3d-e6b8-45ed-b9ec-0de0276b6675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.108460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c0b3d-e6b8-45ed-b9ec-0de0276b6675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:38.110176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c0b3d-e6b8-45ed-b9ec-0de0276b6675 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:38.126420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1238/2490] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:38.133131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:38.143010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:38.153430 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e14097-efbd-4705-8630-f888d775a43d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:38.159994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:38.161911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.162588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.163256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] PATH_INFO: `/v3/projects/7049f613aa064f0b894aa66fff3f4f76/users/9f683c782d984b3fb36d39b13f9e3d9d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.166549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7049f613aa064f0b894aa66fff3f4f76, user_id=9f683c782d984b3fb36d39b13f9e3d9d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:38.170594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:38.184857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1253/2491] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:38 2022] PUT /identity/v3/projects/7049f613aa064f0b894aa66fff3f4f76/users/9f683c782d984b3fb36d39b13f9e3d9d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:38.196827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.197163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.197480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.234571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] MFA Rules not processed for user `9f683c782d984b3fb36d39b13f9e3d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:38.235975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:38.329965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1239/2492] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:38.343044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:38.344946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:38.424641 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da551e-852d-4937-9968-66dc13236155 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:38.429204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a17f-5b71-4967-ae6f-91ab6c47727d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:38.429870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a17f-5b71-4967-ae6f-91ab6c47727d service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.430150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a17f-5b71-4967-ae6f-91ab6c47727d service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.430422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a17f-5b71-4967-ae6f-91ab6c47727d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.443104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a17f-5b71-4967-ae6f-91ab6c47727d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:38.444584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a17f-5b71-4967-ae6f-91ab6c47727d service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:38.457324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1254/2493] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:38.499180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:38.512339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:38.613053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a18a1-0570-48bc-893d-a23eb565ec69 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:38.618691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:38.619824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.620246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.620624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] PATH_INFO: `/v3/users/19482cf9fba54af78d62e6276ce0bf41` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.622090 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19482cf9fba54af78d62e6276ce0bf41)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:38.624205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:38.690312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19482cf9fba54af78d62e6276ce0bf41'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:38.698913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19482cf9fba54af78d62e6276ce0bf41'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:38.709721 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19482cf9fba54af78d62e6276ce0bf41'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:38.727667 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19482cf9fba54af78d62e6276ce0bf41'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:38.728558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19482cf9fba54af78d62e6276ce0bf41'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:38.737418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1240/2494] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:38 2022] DELETE /identity/v3/users/19482cf9fba54af78d62e6276ce0bf41 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:38.743678 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.744036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.744403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.765487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:38.766532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:38.845720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1255/2495] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:38.858555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:38.860357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:38.949354 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f6282-4493-436d-b60d-32299741b9e0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:38.954028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:38.954985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:38.955370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:38.955744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:38.969144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:38.970916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:38.983461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1241/2496] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:35:39.340839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:39.342402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:39.356275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ebfc0-441a-44f1-bb2e-6153f779679d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:39.362346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:39.363357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:39.363640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:39.364593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] PATH_INFO: `/v3/projects/ea9a4d13be1344a2a70cd043f142ae55` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:39.366682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9a4d13be1344a2a70cd043f142ae55)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:39.368892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:39.436476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9a4d13be1344a2a70cd043f142ae55', 'request_id': 'req-7cf63c7b-7ca6-4817-b732-624f09df8e5b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.445719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9a4d13be1344a2a70cd043f142ae55', 'request_id': 'req-7cf63c7b-7ca6-4817-b732-624f09df8e5b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.452685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9a4d13be1344a2a70cd043f142ae55', 'request_id': 'req-7cf63c7b-7ca6-4817-b732-624f09df8e5b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.471840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] The token cache is being invalidate because project ea9a4d13be1344a2a70cd043f142ae55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:39.472142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.472918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.475737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1256/2497] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:39 2022] DELETE /identity/v3/projects/ea9a4d13be1344a2a70cd043f142ae55 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:39.480606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:39.493044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e9d53-ffa0-48da-b452-fe21c3bc53c5 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:39.540236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:39.585268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:39.586181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:39.586459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:39.586731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] PATH_INFO: `/v3/users/9f683c782d984b3fb36d39b13f9e3d9d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:39.588076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f683c782d984b3fb36d39b13f9e3d9d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:39.590776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:39.657527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f683c782d984b3fb36d39b13f9e3d9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.667298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f683c782d984b3fb36d39b13f9e3d9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.679578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f683c782d984b3fb36d39b13f9e3d9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.697868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f683c782d984b3fb36d39b13f9e3d9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.698378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3cc7f79c-f3cb-402c-adc9-7946e7f70dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f683c782d984b3fb36d39b13f9e3d9d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:39.707051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1242/2498] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:39 2022] DELETE /identity/v3/users/9f683c782d984b3fb36d39b13f9e3d9d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:40.099440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.101651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:40.120935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf63c7b-7ca6-4817-b732-624f09df8e5b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.138274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:40.211309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.212254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.212542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.212812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] PATH_INFO: `/v3/projects/7049f613aa064f0b894aa66fff3f4f76` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.214248 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7049f613aa064f0b894aa66fff3f4f76)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.216454 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.291872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7049f613aa064f0b894aa66fff3f4f76', 'request_id': 'req-adf4f6f0-feb9-4ed9-b775-dead731baf4a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:40.301148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7049f613aa064f0b894aa66fff3f4f76', 'request_id': 'req-adf4f6f0-feb9-4ed9-b775-dead731baf4a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:40.319598 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7049f613aa064f0b894aa66fff3f4f76', 'request_id': 'req-adf4f6f0-feb9-4ed9-b775-dead731baf4a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:40.339853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] The token cache is being invalidate because project 7049f613aa064f0b894aa66fff3f4f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:40.340683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:40.341895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-adf4f6f0-feb9-4ed9-b775-dead731baf4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:40.344694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1257/2499] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:40 2022] DELETE /identity/v3/projects/7049f613aa064f0b894aa66fff3f4f76 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:40.351577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.352255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.352255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.376661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:40.377940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:40.452403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1243/2500] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:40.467417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.467830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.468245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.491323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:40.492647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:40.501436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1258/2501] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:40.506954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.524408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df5230-aa28-4639-915a-6c9a26243522 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.532599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.533757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.534255 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.534759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.536701 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:35:40.537258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.540169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.559476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1244/2502] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:35:40.564604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.576004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5d203-814b-4c53-987e-70036e3f339b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.581060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.581745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.582026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.582314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.584346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.586459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.617462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1259/2503] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:40 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:35:40.622370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.635157 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75059e70-fea4-4a34-a9d0-5309ff5635cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.683861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:40.722223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.722926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.723796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.724349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.725170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.727637 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.755393 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1245/2504] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:40 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:40.759712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.771796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02d57-cd12-46ca-9c28-0154d7d57817 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.778045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.779045 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.779538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.780019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.781436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.784555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.798828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1260/2505] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:35:40.804190 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.823012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a73045-000a-4d38-ba1c-d40211c0dd0e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.829159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c8901-da51-4616-b9fe-d527c49b1247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.829896 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c8901-da51-4616-b9fe-d527c49b1247 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.830196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c8901-da51-4616-b9fe-d527c49b1247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.830464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c8901-da51-4616-b9fe-d527c49b1247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.831217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c8901-da51-4616-b9fe-d527c49b1247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.833508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c8901-da51-4616-b9fe-d527c49b1247 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.845848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1246/2506] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:40.851366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:40.863286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5f08b-aba4-4584-991f-9e335e95c239 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:40.869149 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:40.870219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.870563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.870886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] PATH_INFO: `/v3/projects/092016751718433dbeeed347bdae5674/users/f71869995e4a4a2eaca6d89d484c5492/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.889361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] RBAC: Authorizing `identity:create_grant(project_id=092016751718433dbeeed347bdae5674, user_id=f71869995e4a4a2eaca6d89d484c5492, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:40.892067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:40.908755 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1261/2507] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:40 2022] PUT /identity/v3/projects/092016751718433dbeeed347bdae5674/users/f71869995e4a4a2eaca6d89d484c5492/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:40.920762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2433acb-10b2-430c-8773-490b1613414d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:40.921055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2433acb-10b2-430c-8773-490b1613414d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:40.921335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2433acb-10b2-430c-8773-490b1613414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:40.953821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f2433acb-10b2-430c-8773-490b1613414d None None] MFA Rules not processed for user `f71869995e4a4a2eaca6d89d484c5492`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:40.955054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f2433acb-10b2-430c-8773-490b1613414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:41.063030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1247/2508] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:41.068523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.125808 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:41.151947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de544fd5-9246-4008-88c5-2acc7186020f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.158135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.158914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.159273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.159608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.160944 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.164431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.208394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1262/2509] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:41 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:41.212958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2433acb-10b2-430c-8773-490b1613414d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.227837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2433acb-10b2-430c-8773-490b1613414d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.303612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.304559 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.304861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.305162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.305941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.308103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.331457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1248/2510] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:41 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:41.338712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.358443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9b97a-0c34-4081-8780-5c3483f1f98c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.368532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.369304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.369587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.369861 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.370607 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.372298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.387968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1263/2511] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:41.392462 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.408420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82383ef2-1978-4b9f-8452-be95c0cf3c40 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.414610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.415778 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.416143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.416480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] PATH_INFO: `/v3/projects/8788549969174706893ace683ba13cba/users/4767f422f92e48e384fc0c6bf1cbd931/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.435726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8788549969174706893ace683ba13cba, user_id=4767f422f92e48e384fc0c6bf1cbd931, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.438746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.460623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1249/2512] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:41 2022] PUT /identity/v3/projects/8788549969174706893ace683ba13cba/users/4767f422f92e48e384fc0c6bf1cbd931/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:41.467140 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.485921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bab5ef-fb6f-42e3-b56f-271e9c903dde admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.566270 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15926578-ed95-4d96-975b-377c9fd6f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.567657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15926578-ed95-4d96-975b-377c9fd6f36e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.568089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15926578-ed95-4d96-975b-377c9fd6f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.568361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15926578-ed95-4d96-975b-377c9fd6f36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.569124 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15926578-ed95-4d96-975b-377c9fd6f36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.570789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15926578-ed95-4d96-975b-377c9fd6f36e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.582118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1264/2513] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:41.589170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.608380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01df0cb-f092-4c02-9761-d56fe7d92568 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.616357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.617289 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.617571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.617843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] PATH_INFO: `/v3/projects/8788549969174706893ace683ba13cba/users/4767f422f92e48e384fc0c6bf1cbd931/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.620231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8788549969174706893ace683ba13cba, user_id=4767f422f92e48e384fc0c6bf1cbd931, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.622955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.627071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:41.641608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1250/2514] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:41 2022] PUT /identity/v3/projects/8788549969174706893ace683ba13cba/users/4767f422f92e48e384fc0c6bf1cbd931/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:41.653747 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.654246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.654813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.696508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] MFA Rules not processed for user `4767f422f92e48e384fc0c6bf1cbd931`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:41.697762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:41.786991 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1265/2515] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:41.800554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:41.803539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:41.878231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35474b-a7a9-4a19-8dce-6e1bf6913ea2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:41.883152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:41.883866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:41.884183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:41.884461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:41.897416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:41.899253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:41.913159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1251/2516] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:42.033648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:42.046116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc38ce-0714-41dc-a089-8562a528051b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:42.050234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f75e0-067d-4999-b2f0-44ed1d71b5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:42.050902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f75e0-067d-4999-b2f0-44ed1d71b5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:42.052827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f75e0-067d-4999-b2f0-44ed1d71b5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:42.052827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f75e0-067d-4999-b2f0-44ed1d71b5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:42.093859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:42.125410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f75e0-067d-4999-b2f0-44ed1d71b5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:42.126724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f75e0-067d-4999-b2f0-44ed1d71b5e6 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:42.138469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1266/2517] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:44.994557 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:45.085527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c114eee-99ca-467d-a025-4ada89624b94 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:45.091755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:45.092717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:45.093000 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:45.093281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] PATH_INFO: `/v3/users/f71869995e4a4a2eaca6d89d484c5492` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:45.094639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71869995e4a4a2eaca6d89d484c5492)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:45.096751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:45.159510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71869995e4a4a2eaca6d89d484c5492'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:45.171810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71869995e4a4a2eaca6d89d484c5492'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:45.184936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71869995e4a4a2eaca6d89d484c5492'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:45.205949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71869995e4a4a2eaca6d89d484c5492'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:45.206735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71869995e4a4a2eaca6d89d484c5492'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:45.219510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1252/2518] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:44 2022] DELETE /identity/v3/users/f71869995e4a4a2eaca6d89d484c5492 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:45.225256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:45.226151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:45.226151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:45.246691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:45.247935 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:45.327068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1267/2519] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:45.341524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:45.343284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:45.367017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:45.419063 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe581f4a-6a43-4243-bf03-27f470a8d028 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:45.423989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:45.424900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:45.425240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:45.425586 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:45.437447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:45.438931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:45.453062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1253/2520] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:35:45.931895 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:45.934625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:45.949347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7f7c4-61b7-4019-8ed8-b321b9d6b4ec None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:45.954923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:45.955858 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:45.956174 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:45.956565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] PATH_INFO: `/v3/projects/092016751718433dbeeed347bdae5674` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:45.958878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=092016751718433dbeeed347bdae5674)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:45.962754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:46.029896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '092016751718433dbeeed347bdae5674', 'request_id': 'req-667fae7b-e361-4b9b-807c-d21e54eddc76'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.039625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '092016751718433dbeeed347bdae5674', 'request_id': 'req-667fae7b-e361-4b9b-807c-d21e54eddc76'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.046605 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '092016751718433dbeeed347bdae5674', 'request_id': 'req-667fae7b-e361-4b9b-807c-d21e54eddc76'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.063566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] The token cache is being invalidate because project 092016751718433dbeeed347bdae5674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:46.063907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.064547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.066146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1268/2521] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:45 2022] DELETE /identity/v3/projects/092016751718433dbeeed347bdae5674 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:46.072040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:46.084734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be48e9-96c2-4ca3-ac83-2a69fd58ab32 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:46.162878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:46.164552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:46.165024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:46.165538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] PATH_INFO: `/v3/users/4767f422f92e48e384fc0c6bf1cbd931` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:46.167846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4767f422f92e48e384fc0c6bf1cbd931)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:46.171636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:46.254670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4767f422f92e48e384fc0c6bf1cbd931'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.262219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4767f422f92e48e384fc0c6bf1cbd931'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.276551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4767f422f92e48e384fc0c6bf1cbd931'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.307196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4767f422f92e48e384fc0c6bf1cbd931'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.308443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-36ee38f5-23a7-42dd-a720-454083f159fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4767f422f92e48e384fc0c6bf1cbd931'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.320253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1254/2522] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:46 2022] DELETE /identity/v3/users/4767f422f92e48e384fc0c6bf1cbd931 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:46.714475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:46.716402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:46.732804 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fae7b-e361-4b9b-807c-d21e54eddc76 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:46.810695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:46.811672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:46.812001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:46.812381 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] PATH_INFO: `/v3/projects/8788549969174706893ace683ba13cba` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:46.813923 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8788549969174706893ace683ba13cba)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:46.815981 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:46.819009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:46.879154 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8788549969174706893ace683ba13cba', 'request_id': 'req-a684562e-3d01-4347-9ba4-67ab31968e5e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.888079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8788549969174706893ace683ba13cba', 'request_id': 'req-a684562e-3d01-4347-9ba4-67ab31968e5e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.895523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8788549969174706893ace683ba13cba', 'request_id': 'req-a684562e-3d01-4347-9ba4-67ab31968e5e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.911427 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] The token cache is being invalidate because project 8788549969174706893ace683ba13cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:46.912011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.912740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a684562e-3d01-4347-9ba4-67ab31968e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:46.914360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1269/2523] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:46 2022] DELETE /identity/v3/projects/8788549969174706893ace683ba13cba => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:46.920605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:46.921132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:46.921606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:46.952785 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:46.954916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:47.012724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:47.049223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1255/2524] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:47.060207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.060634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.060634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.081575 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:47.083670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:47.093095 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1270/2525] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:47.097645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.110831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa7fd-9749-40e1-ba17-931914e4e2cb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.119073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.120338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.120841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.121343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.123060 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:35:47.123728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.127083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.151708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1256/2526] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:35:47.157219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.169854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d118e7e-c17e-4cae-88fb-5c2ab635fe57 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.175542 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.176417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.176830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.177111 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.177840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.179809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.215866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1271/2527] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:47 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:35:47.222202 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.234437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ae51f-b549-42f0-9fbc-57a4c0c0eba4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.320573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.321176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.321461 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.321736 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.322459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.325838 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.352807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1257/2528] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:47 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:47.357773 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.370555 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e89997-2742-4800-957c-15d22d1d7049 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.375893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.376632 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.376914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.377190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.377951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.380258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.393417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1272/2529] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:47.398811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.409886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407445db-a042-4d74-bc9d-e9a96649a465 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.416761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceec7e3-04c1-498b-8947-5d9859717539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.417475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec7e3-04c1-498b-8947-5d9859717539 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.417759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec7e3-04c1-498b-8947-5d9859717539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.418038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec7e3-04c1-498b-8947-5d9859717539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.418971 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec7e3-04c1-498b-8947-5d9859717539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.420988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec7e3-04c1-498b-8947-5d9859717539 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.434133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1258/2530] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:35:47.441027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.460707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970798e5-4129-4e22-aff4-9f4528d6ede9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.469157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.470819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.471305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.471803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] PATH_INFO: `/v3/projects/cff1bdf2a57f4eaa9a6eea326dd28f84/users/4af7dcc64ca74131a7dfde78245ddbd0/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.500951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff1bdf2a57f4eaa9a6eea326dd28f84, user_id=4af7dcc64ca74131a7dfde78245ddbd0, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.507402 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.530297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1273/2531] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:47 2022] PUT /identity/v3/projects/cff1bdf2a57f4eaa9a6eea326dd28f84/users/4af7dcc64ca74131a7dfde78245ddbd0/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:47.548660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.548965 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.549215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.585386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] MFA Rules not processed for user `4af7dcc64ca74131a7dfde78245ddbd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:47.587121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:47.711194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1259/2532] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:47.717403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.802572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89e6de-8304-4aad-90b4-d205a38ce222 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.809587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.810567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.810950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.811329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.812184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.815085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.856950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1274/2533] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:47 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:47.861351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:47.875507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f827-b2a9-45db-832e-0c0694af7606 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:47.916419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:47.951651 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:47.952779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:47.953140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:47.953887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:47.954793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:47.957379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:47.982376 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1260/2534] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:47 2022] POST /identity/v3/users => generated 296 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:35:47.986834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:48.008582 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1faf8f3-a8a9-4299-b095-78a720baecca admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:48.016803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:48.017915 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:48.018424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:48.018907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:48.020293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:48.023359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:48.033548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:48.044075 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1275/2535] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:35:48.048902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:48.061564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbd1c-fbe3-44cd-b543-a2acfe177d9e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:48.066678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:48.068427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:48.068720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:48.069002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] PATH_INFO: `/v3/projects/9a1d812823f543fe86f48257512ff19e/users/bb3fc95428444e2f92111f066592aa57/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:48.089383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1d812823f543fe86f48257512ff19e, user_id=bb3fc95428444e2f92111f066592aa57, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:48.093922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:48.115047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1261/2536] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:48 2022] PUT /identity/v3/projects/9a1d812823f543fe86f48257512ff19e/users/bb3fc95428444e2f92111f066592aa57/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:48.119312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:48.132366 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824fe0b-9e00-4dd1-8e02-4341579d6e8a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:48.227909 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc52bb-294a-4d8b-a292-7b61c56dd07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:48.229117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc52bb-294a-4d8b-a292-7b61c56dd07f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:48.229628 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc52bb-294a-4d8b-a292-7b61c56dd07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:48.230131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc52bb-294a-4d8b-a292-7b61c56dd07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:48.231798 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc52bb-294a-4d8b-a292-7b61c56dd07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:48.234778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc52bb-294a-4d8b-a292-7b61c56dd07f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:48.256216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1276/2537] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:35:48.260609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:48.273720 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a09f3-ac1b-412f-a795-87c16856bd76 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:48.280612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:48.281642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:48.281930 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:48.282212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] PATH_INFO: `/v3/projects/9a1d812823f543fe86f48257512ff19e/users/bb3fc95428444e2f92111f066592aa57/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:48.284780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1d812823f543fe86f48257512ff19e, user_id=bb3fc95428444e2f92111f066592aa57, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:48.288100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:48.302677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1262/2538] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:48 2022] PUT /identity/v3/projects/9a1d812823f543fe86f48257512ff19e/users/bb3fc95428444e2f92111f066592aa57/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:48.314231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:48.314681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:48.315108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:48.347763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] MFA Rules not processed for user `bb3fc95428444e2f92111f066592aa57`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:48.349110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:48.435976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1277/2539] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:48.451097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:48.453048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:48.536507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626728-7c10-4d4f-b939-c4eb6fb762b1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:48.540594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960472c-c376-4f93-ac28-72edce57b7b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:48.541284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960472c-c376-4f93-ac28-72edce57b7b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:48.541569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960472c-c376-4f93-ac28-72edce57b7b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:48.541845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960472c-c376-4f93-ac28-72edce57b7b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:48.553546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960472c-c376-4f93-ac28-72edce57b7b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:48.555041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960472c-c376-4f93-ac28-72edce57b7b5 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:48.567367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1263/2540] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:49.092956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:49.174965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89cf30-1127-4bd3-9f36-4b1a03cb43f5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:49.181206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:49.182316 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:49.182676 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:49.183029 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] PATH_INFO: `/v3/users/4af7dcc64ca74131a7dfde78245ddbd0` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:49.185338 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af7dcc64ca74131a7dfde78245ddbd0)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:49.188369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:49.220302 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:49.257705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af7dcc64ca74131a7dfde78245ddbd0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:49.275970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af7dcc64ca74131a7dfde78245ddbd0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:49.287223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af7dcc64ca74131a7dfde78245ddbd0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:49.288194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af7dcc64ca74131a7dfde78245ddbd0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:49.289040 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af7dcc64ca74131a7dfde78245ddbd0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:49.306185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1278/2541] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:49 2022] DELETE /identity/v3/users/4af7dcc64ca74131a7dfde78245ddbd0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:35:49.313213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:49.313865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:49.313865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:49.334746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:49.336029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:49.415409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1264/2542] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:49.427685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:49.429458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:49.502501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065f551-6c6e-4053-b9de-efc4f391e376 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:49.507781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:49.508529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:49.508818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:49.509117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:49.521975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:49.523545 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:49.534962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1279/2543] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:35:49.941946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:49.945378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:49.997581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9ede-198c-45f6-89be-354da16c7d80 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:50.008538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:50.009800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:50.010246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:50.010711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] PATH_INFO: `/v3/projects/cff1bdf2a57f4eaa9a6eea326dd28f84` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:50.012799 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff1bdf2a57f4eaa9a6eea326dd28f84)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:50.016384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:50.059339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:50.093767 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff1bdf2a57f4eaa9a6eea326dd28f84', 'request_id': 'req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.102116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff1bdf2a57f4eaa9a6eea326dd28f84', 'request_id': 'req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.109906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff1bdf2a57f4eaa9a6eea326dd28f84', 'request_id': 'req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.127254 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] The token cache is being invalidate because project cff1bdf2a57f4eaa9a6eea326dd28f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:50.127713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.128691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.131745 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1265/2544] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:49 2022] DELETE /identity/v3/projects/cff1bdf2a57f4eaa9a6eea326dd28f84 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:50.136348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:50.148201 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3527-5511-49cf-badd-38e512290241 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:50.245613 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:50.247597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:50.248125 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:50.248625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] PATH_INFO: `/v3/users/bb3fc95428444e2f92111f066592aa57` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:50.250970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3fc95428444e2f92111f066592aa57)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:50.255683 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:50.350227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:50.352711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3fc95428444e2f92111f066592aa57'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.376410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3fc95428444e2f92111f066592aa57'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.387965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3fc95428444e2f92111f066592aa57'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.388914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3fc95428444e2f92111f066592aa57'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.389647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d25ba74a-378f-4f25-a84b-ca4e27ed8dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3fc95428444e2f92111f066592aa57'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:50.407460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1280/2545] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:35:50 2022] DELETE /identity/v3/users/bb3fc95428444e2f92111f066592aa57 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:50.778423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:50.780590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:50.806567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcffdcc-ba6d-451c-b5a4-605b3bc7ea34 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:50.927872 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:50.929495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:50.930002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:50.930509 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] PATH_INFO: `/v3/projects/9a1d812823f543fe86f48257512ff19e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:50.933209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1d812823f543fe86f48257512ff19e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:50.936851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.023834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1d812823f543fe86f48257512ff19e', 'request_id': 'req-92e356bd-f629-433a-81b6-51d3248f92cb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:51.030663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1d812823f543fe86f48257512ff19e', 'request_id': 'req-92e356bd-f629-433a-81b6-51d3248f92cb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:51.043588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1d812823f543fe86f48257512ff19e', 'request_id': 'req-92e356bd-f629-433a-81b6-51d3248f92cb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:51.062549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] The token cache is being invalidate because project 9a1d812823f543fe86f48257512ff19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:35:51.062837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:51.063525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-92e356bd-f629-433a-81b6-51d3248f92cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:35:51.065230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1266/2546] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:35:50 2022] DELETE /identity/v3/projects/9a1d812823f543fe86f48257512ff19e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:51.072361 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.072874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.073356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.096925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:51.098435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:51.201540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1281/2547] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:35:51.211523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.211813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.212139 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.237760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:51.240269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:51.248811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1267/2548] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:51.253358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.265913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5d524-bdfa-4b9d-9c0c-ab9302899173 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:51.271911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:51.272650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.272963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.273240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.274004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:35:51.274311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:51.276091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.294752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1282/2549] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:35:51.302204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.328381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb918cf-ce53-4ea0-b0de-7b0725d3de74 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:51.338480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:51.339670 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.340245 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.340793 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.342270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:51.346020 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.394625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1268/2550] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:35:51 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:51.401666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.414647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33afa-dc45-4916-880e-5a67765eb743 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:51.484405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:51.485093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.485371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.485644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.486393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:51.488421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.518615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1283/2551] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:35:51 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:35:51.524836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.538622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d8393-db9c-4574-ac2f-2b43cff16e2c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:51.544870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:51.545683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.546073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.546455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.547429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:51.549489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.562866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1269/2552] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:51.568496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.579418 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839cb7e-1327-43ff-8120-0123652e8197 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:51.584946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb8c8c-4294-4378-8768-f79df497e768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:51.585654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb8c8c-4294-4378-8768-f79df497e768 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.586143 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb8c8c-4294-4378-8768-f79df497e768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.586344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb8c8c-4294-4378-8768-f79df497e768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.587004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb8c8c-4294-4378-8768-f79df497e768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:51.588695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb8c8c-4294-4378-8768-f79df497e768 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.602710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1284/2553] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:35:51.609591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.617642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:51.623837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5050e-1770-46ac-a0a5-9aa65666459e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:51.629834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:51.631677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.632028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.632522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] PATH_INFO: `/v3/projects/2a018c9d73cc466099993b27c5b9ea61/users/be6e4e550ac54e1fb01f820861c48ce5/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.669318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a018c9d73cc466099993b27c5b9ea61, user_id=be6e4e550ac54e1fb01f820861c48ce5, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:51.679451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:51.702370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1270/2554] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:35:51 2022] PUT /identity/v3/projects/2a018c9d73cc466099993b27c5b9ea61/users/be6e4e550ac54e1fb01f820861c48ce5/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:35:51.714410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:51.714985 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:51.716168 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:51.760207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] MFA Rules not processed for user `be6e4e550ac54e1fb01f820861c48ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:35:51.762424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:51.797681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:35:51.888938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1285/2555] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:35:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:35:51.903454 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:35:51.905695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:35:52.001204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dc4e3-d103-44ef-82c4-f9bf0502d6bf admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:35:52.007622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:35:52.008865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:35:52.009384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:35:52.009935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:35:52.028542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:35:52.031361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:35:52.053956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1271/2556] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:35:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:01.154625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:01.156537 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:01.277584 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6202625-e127-474c-8b7d-1ff08884919e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:01.283872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:01.285099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:01.286230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:01.286710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:01.289906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:01.413649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:01.416294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:01.436031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1286/2557] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:02.043868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:06.364873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:06.367677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:06.393722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003115-6de7-40b2-9cd0-f892ce7fb460 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:06.403518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:06.405214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:06.405717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:06.406483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] PATH_INFO: `/v3/users/73dc3ab6bc6448a89b26e3e1a1b5c6e8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:06.408876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73dc3ab6bc6448a89b26e3e1a1b5c6e8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:06.413415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:06.501306 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73dc3ab6bc6448a89b26e3e1a1b5c6e8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:06.530719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73dc3ab6bc6448a89b26e3e1a1b5c6e8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:06.543258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73dc3ab6bc6448a89b26e3e1a1b5c6e8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:06.564904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73dc3ab6bc6448a89b26e3e1a1b5c6e8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:06.565868 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4a92c92f-c929-4350-b248-2741867dc702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73dc3ab6bc6448a89b26e3e1a1b5c6e8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:06.574676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1272/2558] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:06 2022] DELETE /identity/v3/users/73dc3ab6bc6448a89b26e3e1a1b5c6e8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:06.834288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:06.853538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750902-ab8d-4f92-be4c-6d8fda53c7ad service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:06.934172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:06.935063 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:06.935352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:06.935649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] PATH_INFO: `/v3/projects/7904e707e39846fdb8b7d019719dc163` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:06.937358 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7904e707e39846fdb8b7d019719dc163)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:06.939372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:06.954018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:07.039386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7904e707e39846fdb8b7d019719dc163', 'request_id': 'req-19ad5cbc-7cc7-42a3-9146-6503c70e5823'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:07.052388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7904e707e39846fdb8b7d019719dc163', 'request_id': 'req-19ad5cbc-7cc7-42a3-9146-6503c70e5823'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:07.071259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7904e707e39846fdb8b7d019719dc163', 'request_id': 'req-19ad5cbc-7cc7-42a3-9146-6503c70e5823'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:07.103620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] The token cache is being invalidate because project 7904e707e39846fdb8b7d019719dc163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:07.104131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:07.105005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-19ad5cbc-7cc7-42a3-9146-6503c70e5823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:07.107412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1287/2559] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:06 2022] DELETE /identity/v3/projects/7904e707e39846fdb8b7d019719dc163 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:07.116114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.116442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.116707 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.137755 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:07.139032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:07.220931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1273/2560] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:07.233480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.233480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.233805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.262825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:07.264847 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:07.277824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1288/2561] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:07.282699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.294420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e51e7d-6c4c-4719-9f21-250eea16112a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:07.299908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:07.301434 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.301717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.301998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.302743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:07.303055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:07.305130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:07.319573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1274/2562] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:07.325085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.338278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0cd65-4ec6-4758-af11-7e6a5171c23a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:07.345311 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:07.346336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.346786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.347343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.350164 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:07.352998 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:07.425053 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1289/2563] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:07 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:07.428278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.446714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0e4c3-04b6-4c88-9603-a0c8a0263f4f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:07.562341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:07.563487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.563955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.564424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.565344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:07.568263 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:07.577539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:07.600257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1275/2564] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:07 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:07.605538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.626631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd246c8-211e-4ee8-96a7-0140d4d006d5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:07.637322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:07.637322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.637322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.637322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.638084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:07.638084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:07.657031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1290/2565] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:07.662698 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.676904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab53ac-ca37-411b-82ac-55d1b631f70d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:07.682913 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173f97-996e-4187-af6c-04dda03177c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:07.684297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f97-996e-4187-af6c-04dda03177c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.684696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f97-996e-4187-af6c-04dda03177c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.685070 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f97-996e-4187-af6c-04dda03177c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.686040 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72173f97-996e-4187-af6c-04dda03177c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:07.687806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72173f97-996e-4187-af6c-04dda03177c5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:07.703284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1276/2566] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:07.710544 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.724531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ecc7d-0042-4ee4-8952-6ae271d19988 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:07.732721 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:07.733644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.733922 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.734233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] PATH_INFO: `/v3/projects/91efbe839c644278a79556ae257a9fd8/users/939ecba77c3b4caf9d6908a71f83f66c/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.755127 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91efbe839c644278a79556ae257a9fd8, user_id=939ecba77c3b4caf9d6908a71f83f66c, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:07.758746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:07.774754 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1291/2567] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:07 2022] PUT /identity/v3/projects/91efbe839c644278a79556ae257a9fd8/users/939ecba77c3b4caf9d6908a71f83f66c/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:07.787897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:07.788193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:07.788470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:07.823227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] MFA Rules not processed for user `939ecba77c3b4caf9d6908a71f83f66c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:07.824512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:07.924207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1277/2568] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:07.937445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:07.939230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:08.037601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af0bc-9bf8-4ed8-9bcb-f938864d0a8e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:08.045774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965ab8f-c95c-4842-8b1c-b0fe88618270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:08.045774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965ab8f-c95c-4842-8b1c-b0fe88618270 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:08.045774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965ab8f-c95c-4842-8b1c-b0fe88618270 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:08.046302 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965ab8f-c95c-4842-8b1c-b0fe88618270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:08.055621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965ab8f-c95c-4842-8b1c-b0fe88618270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:08.057297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965ab8f-c95c-4842-8b1c-b0fe88618270 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:08.068565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1292/2569] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:11.043087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:11.160235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd88db-cf1c-40fd-8c0c-c576069d2d5e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:11.169430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:11.170411 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:11.170813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:11.171371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] PATH_INFO: `/v3/users/939ecba77c3b4caf9d6908a71f83f66c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:11.173182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] RBAC: Authorizing `identity:delete_user(user_id=939ecba77c3b4caf9d6908a71f83f66c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:11.176493 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:11.264747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939ecba77c3b4caf9d6908a71f83f66c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.280221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939ecba77c3b4caf9d6908a71f83f66c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.291996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939ecba77c3b4caf9d6908a71f83f66c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.304568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:11.318140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939ecba77c3b4caf9d6908a71f83f66c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.319638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939ecba77c3b4caf9d6908a71f83f66c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.329525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1278/2570] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:11 2022] DELETE /identity/v3/users/939ecba77c3b4caf9d6908a71f83f66c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:11.334549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:11.335091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:11.335091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:11.358130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:11.360696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:11.380399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:11.422340 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:11.424924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:11.455420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1293/2571] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:11.468618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:11.471126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:11.534878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa3b1b-2171-44a4-a8ae-092de1df114d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:11.543841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:11.544845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1af61e-5b9b-42ec-9664-30cc5beea8e8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:11.545457 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:11.545988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:11.546503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] PATH_INFO: `/v3/users/be6e4e550ac54e1fb01f820861c48ce5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:11.549326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d9169-f9e4-4316-a379-ff90abb4c9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:11.549520 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6e4e550ac54e1fb01f820861c48ce5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:11.550008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d9169-f9e4-4316-a379-ff90abb4c9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:11.550289 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d9169-f9e4-4316-a379-ff90abb4c9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:11.550564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d9169-f9e4-4316-a379-ff90abb4c9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:11.553829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:11.562629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d9169-f9e4-4316-a379-ff90abb4c9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:11.564113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d9169-f9e4-4316-a379-ff90abb4c9d5 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:11.576412 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1294/2572] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:11.645537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6e4e550ac54e1fb01f820861c48ce5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.659934 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6e4e550ac54e1fb01f820861c48ce5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.674535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6e4e550ac54e1fb01f820861c48ce5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.701186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6e4e550ac54e1fb01f820861c48ce5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.702408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6e4e550ac54e1fb01f820861c48ce5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:11.714614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1279/2573] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:11 2022] DELETE /identity/v3/users/be6e4e550ac54e1fb01f820861c48ce5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:11.720696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:11.721166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:11.721569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:11.755337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:11.757382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:11.844951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1295/2574] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:11.857541 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:11.859079 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:11.940565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965aed2-4374-43a5-be86-4de4018cffe5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:11.947312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:11.948535 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:11.949037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:11.949539 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:11.970372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:11.974490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:11.992970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1280/2575] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:12.289774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:12.292241 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.311234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a139c8c-3049-457b-8e5e-eca73cb62231 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:12.325329 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:12.326968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.327377 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.327779 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] PATH_INFO: `/v3/projects/91efbe839c644278a79556ae257a9fd8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.331948 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91efbe839c644278a79556ae257a9fd8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:12.337708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:12.429367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91efbe839c644278a79556ae257a9fd8', 'request_id': 'req-bce404cd-7c0b-41d5-87cd-068866e1d793'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.439356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91efbe839c644278a79556ae257a9fd8', 'request_id': 'req-bce404cd-7c0b-41d5-87cd-068866e1d793'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.450074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:12.455299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91efbe839c644278a79556ae257a9fd8', 'request_id': 'req-bce404cd-7c0b-41d5-87cd-068866e1d793'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.459496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:12.462579 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.485280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] The token cache is being invalidate because project 91efbe839c644278a79556ae257a9fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:12.486015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.487090 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bce404cd-7c0b-41d5-87cd-068866e1d793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.488072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3739e-118b-49ae-8d18-bc19b82d8cb0 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:12.491369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1296/2576] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:12 2022] DELETE /identity/v3/projects/91efbe839c644278a79556ae257a9fd8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:12.498301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3b10d-6405-4ab2-84a0-ec9818f8027e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.498920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3b10d-6405-4ab2-84a0-ec9818f8027e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.499211 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:12.500251 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3b10d-6405-4ab2-84a0-ec9818f8027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.530041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-32c3b10d-6405-4ab2-84a0-ec9818f8027e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:12.531428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-32c3b10d-6405-4ab2-84a0-ec9818f8027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.611429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:12.613100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.613656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.614126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] PATH_INFO: `/v3/projects/2a018c9d73cc466099993b27c5b9ea61` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.616276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a018c9d73cc466099993b27c5b9ea61)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:12.619827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:12.627268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1297/2577] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:12.631621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.632604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.632604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.658554 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:12.660585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.672009 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1298/2578] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:12.678523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:12.689849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a018c9d73cc466099993b27c5b9ea61', 'request_id': 'req-fc923860-fc48-476c-9693-272d84cfa724'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.690325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f46d58-0143-48a7-813b-8317995ddb2b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:12.698054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a018c9d73cc466099993b27c5b9ea61', 'request_id': 'req-fc923860-fc48-476c-9693-272d84cfa724'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.709449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a018c9d73cc466099993b27c5b9ea61', 'request_id': 'req-fc923860-fc48-476c-9693-272d84cfa724'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.727538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] The token cache is being invalidate because project 2a018c9d73cc466099993b27c5b9ea61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:12.728152 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.729110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc923860-fc48-476c-9693-272d84cfa724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:12.731080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1281/2579] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:12 2022] DELETE /identity/v3/projects/2a018c9d73cc466099993b27c5b9ea61 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:12.736768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4efc2-c3a2-4294-a3b5-c3b5bae3c61c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.737342 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4efc2-c3a2-4294-a3b5-c3b5bae3c61c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.737743 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4efc2-c3a2-4294-a3b5-c3b5bae3c61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.761622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-edf4efc2-c3a2-4294-a3b5-c3b5bae3c61c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:12.763168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-edf4efc2-c3a2-4294-a3b5-c3b5bae3c61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.774732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:12.775612 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.776073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.776515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.777497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:12.778014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:12.780303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:12.797177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1299/2580] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:12.804439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:12.806399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.821119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51f858-0e80-4571-998b-fd2f451e5a28 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:12.826305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:12.826992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.827279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.827608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.828509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:12.830884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:12.856656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1282/2581] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:12.865699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.865989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:12.866278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:12.872511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1300/2582] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:12 2022] POST /identity/v3/projects => generated 403 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:12.879487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:12.893421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:12.895209 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:12.939818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:12.970393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:12.982700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1283/2583] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:12.990343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:12.990626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55faa9-b703-4f1a-891f-522a59f1022c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:12.998677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:12.999564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:12.999992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.000477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.001395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.004874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7117fd9-96bf-4889-a207-a0ce5cbdc7c3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.005737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.010773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.011624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.012092 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.012747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.013928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:13.014449 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.016893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.036869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1301/2584] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:12 2022] POST /identity/v3/users => generated 302 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:13.037137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1284/2585] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:13.038569 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.041806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.052572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdd5a7-a5b3-4d74-8f72-7c50edbed1cd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.055183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9e52f-af2b-4a56-bf5a-5470e4838c99 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.058661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.059506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.059884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.060389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.061021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.061475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.061960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.062476 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.062817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.063602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.064027 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.066177 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.085979 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1302/2586] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:13.087782 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.107508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceac152-932c-4a52-a67b-2a24a0b671c9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.110733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1285/2587] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:13 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:13.116560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.196277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.197375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.197849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.198303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.199456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.202538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.217217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37353-e3e4-42f4-b614-470c0b2527cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.219766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1303/2588] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:13.225692 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.226426 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.226712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.227097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.227512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.228684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.230774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.237945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d53f9-cdc9-4b51-a6ee-4e4fde514273 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.245588 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a2ff0-af04-4c0a-be66-38943efd86ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.246878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a2ff0-af04-4c0a-be66-38943efd86ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.247281 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a2ff0-af04-4c0a-be66-38943efd86ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.247831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a2ff0-af04-4c0a-be66-38943efd86ac admin admin] PATH_INFO: `/v3/projects/bcd40164935946679b6cdccf97f5aa54/users/ee436e8e52754c458a966cf8615fda54/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.264178 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1286/2589] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:13 2022] POST /identity/v3/users => generated 285 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:13.270023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.272611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a2ff0-af04-4c0a-be66-38943efd86ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd40164935946679b6cdccf97f5aa54, user_id=ee436e8e52754c458a966cf8615fda54, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.276013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a2ff0-af04-4c0a-be66-38943efd86ac admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.283724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ab771-3723-402e-9f68-7f2c04410da9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.292136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.293130 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.293676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.294042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.295355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.297737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1304/2590] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:13 2022] PUT /identity/v3/projects/bcd40164935946679b6cdccf97f5aa54/users/ee436e8e52754c458a966cf8615fda54/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:13.298136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.311002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1287/2591] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:13.314246 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.314699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.315071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.316027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.330066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd48142-3cf2-43a1-9a9c-239629149e48 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.356969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] MFA Rules not processed for user `ee436e8e52754c458a966cf8615fda54`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:13.359107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:13.404967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.405815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.406205 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.406626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.407700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.409845 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.424969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1288/2592] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:13.430121 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.444030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8d60d-d583-4e72-8869-191b7a05dec3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.450898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3f7b-c3a0-4416-b22f-b91d065801f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.452280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3f7b-c3a0-4416-b22f-b91d065801f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.452691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3f7b-c3a0-4416-b22f-b91d065801f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.453084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3f7b-c3a0-4416-b22f-b91d065801f8 admin admin] PATH_INFO: `/v3/projects/25d292c37a034d23803e395980b4219a/users/090960f27da74a65b8eeb82be1d31ac2/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.460444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1305/2593] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:13.465737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.478324 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3f7b-c3a0-4416-b22f-b91d065801f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d292c37a034d23803e395980b4219a, user_id=090960f27da74a65b8eeb82be1d31ac2, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.478933 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e97be-9845-4e10-a38a-863d1ebcc25a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.482338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3f7b-c3a0-4416-b22f-b91d065801f8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.484395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.485617 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.486073 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.487000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.487524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:13.488965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.491978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.505029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1289/2594] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:13 2022] PUT /identity/v3/projects/25d292c37a034d23803e395980b4219a/users/090960f27da74a65b8eeb82be1d31ac2/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:13.516262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.516666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.517244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.531240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1306/2595] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:13 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:13.538752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.557321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:13.559852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] MFA Rules not processed for user `090960f27da74a65b8eeb82be1d31ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:13.561645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:13.651836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773af1d-1178-4d77-bcc7-b8e281617921 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.660936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.661633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.661947 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.662375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.663192 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.665245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.669909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1290/2596] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:13.690451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.694159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:13.694703 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1307/2597] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:13 2022] POST /identity/v3/users => generated 309 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:13.700262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.720911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9446e7-0360-4d81-a76c-8720c4f2d172 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.731278 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.732702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.732990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.733468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.734856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.737999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.762110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1308/2598] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:13.763696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.779202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0da66-ba67-4121-a9dd-7f7b348ad7a7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.788296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.789838 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.790301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.801264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] PATH_INFO: `/v3/projects/82ef09ad8e3846668bbe3e8d314f00b6/users/5468e0235e87440d818077704568c832/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.801566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2aab-28fd-4955-a1f0-e1ccfc038611 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.806564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80ea2163e08f4443a3d6af03a59351f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.807576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.807873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.808250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.820024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.821680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.827766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ef09ad8e3846668bbe3e8d314f00b6, user_id=5468e0235e87440d818077704568c832, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.833429 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.834589 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1291/2599] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:13.857066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1309/2600] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:13 2022] PUT /identity/v3/projects/82ef09ad8e3846668bbe3e8d314f00b6/users/5468e0235e87440d818077704568c832/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:13.862384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.876876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0eae1-55c9-4ac0-84bc-cea4cb06a884 service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:13.956699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad6fb-75a0-4103-820a-ef25039ab654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:13.958798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad6fb-75a0-4103-820a-ef25039ab654 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:13.959031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad6fb-75a0-4103-820a-ef25039ab654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:13.959391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad6fb-75a0-4103-820a-ef25039ab654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:13.960360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad6fb-75a0-4103-820a-ef25039ab654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:13.962004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad6fb-75a0-4103-820a-ef25039ab654 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:13.976146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1292/2601] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:13.979769 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:13.998096 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd9878-6256-4a2d-bb6e-988a234d27ee admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:14.006954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:14.007878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:14.008189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:14.008474 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] PATH_INFO: `/v3/projects/82ef09ad8e3846668bbe3e8d314f00b6/users/5468e0235e87440d818077704568c832/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:14.011672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ef09ad8e3846668bbe3e8d314f00b6, user_id=5468e0235e87440d818077704568c832, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:14.014579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:14.035837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1310/2602] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:13 2022] PUT /identity/v3/projects/82ef09ad8e3846668bbe3e8d314f00b6/users/5468e0235e87440d818077704568c832/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:14.054415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:14.054936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:14.055452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:14.096957 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] MFA Rules not processed for user `5468e0235e87440d818077704568c832`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:14.099486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:14.114628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:14.193582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1293/2603] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:14.205370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:14.208531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:14.278649 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44590b8c-74d5-4d10-8f92-9888b725bf3c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:14.284637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4552db-c114-4f86-b123-5cd464af8d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:14.286450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4552db-c114-4f86-b123-5cd464af8d1d service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:14.286846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4552db-c114-4f86-b123-5cd464af8d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:14.287223 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4552db-c114-4f86-b123-5cd464af8d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:14.303425 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4552db-c114-4f86-b123-5cd464af8d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:14.306249 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4552db-c114-4f86-b123-5cd464af8d1d service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:14.325491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1311/2604] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:14 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:14.609364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:14.697695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11988176-c640-4007-a2bd-ddf1b6039544 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:14.703988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:14.704949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:14.705250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:14.705521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] PATH_INFO: `/v3/users/ee436e8e52754c458a966cf8615fda54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:14.707472 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee436e8e52754c458a966cf8615fda54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:14.710146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:14.776842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee436e8e52754c458a966cf8615fda54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:14.784754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee436e8e52754c458a966cf8615fda54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:14.794386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee436e8e52754c458a966cf8615fda54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:14.819765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee436e8e52754c458a966cf8615fda54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:14.820645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee436e8e52754c458a966cf8615fda54'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:14.829287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1294/2605] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:14 2022] DELETE /identity/v3/users/ee436e8e52754c458a966cf8615fda54 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:14.834231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:14.835034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:14.835034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:14.865854 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:14.868211 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:14.936964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:14.958814 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1312/2606] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:14.966264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:14.967703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:15.014129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:15.069260 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f545b96-529e-4fb0-9852-b21ca3a8b581 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:15.086269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:15.087470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:15.087883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:15.088409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:15.105525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:15.112966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:15.132099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1295/2607] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:15.615153 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:15.618672 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:15.660172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80d965-f797-4887-b3ff-a5207b8f18d4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:15.668955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:15.669977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:15.670442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:15.670824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] PATH_INFO: `/v3/projects/bcd40164935946679b6cdccf97f5aa54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:15.672296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd40164935946679b6cdccf97f5aa54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:15.674729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:15.743245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd40164935946679b6cdccf97f5aa54', 'request_id': 'req-1bbb1e94-9210-4edb-9506-a29ed2433ce2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:15.756225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd40164935946679b6cdccf97f5aa54', 'request_id': 'req-1bbb1e94-9210-4edb-9506-a29ed2433ce2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:15.763908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd40164935946679b6cdccf97f5aa54', 'request_id': 'req-1bbb1e94-9210-4edb-9506-a29ed2433ce2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:15.789253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] The token cache is being invalidate because project bcd40164935946679b6cdccf97f5aa54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:15.789710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:15.790821 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:15.793445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1313/2608] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:15 2022] DELETE /identity/v3/projects/bcd40164935946679b6cdccf97f5aa54 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:15.800010 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:15.819928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376044b-d331-4e07-9880-49738f502aa9 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:15.928494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:15.930409 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:15.931068 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:15.931555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] PATH_INFO: `/v3/users/5468e0235e87440d818077704568c832` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:15.934293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5468e0235e87440d818077704568c832)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:15.938558 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:16.010147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5468e0235e87440d818077704568c832'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.029591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5468e0235e87440d818077704568c832'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.041956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5468e0235e87440d818077704568c832'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.069883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5468e0235e87440d818077704568c832'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.070832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a3a8ef86-4f35-4b06-8256-ddc028edcd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5468e0235e87440d818077704568c832'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.079221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1296/2609] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:15 2022] DELETE /identity/v3/users/5468e0235e87440d818077704568c832 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:16.539457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:16.541758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:16.566131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb1e94-9210-4edb-9506-a29ed2433ce2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:16.653897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:16.654785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:16.655066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:16.655340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] PATH_INFO: `/v3/projects/82ef09ad8e3846668bbe3e8d314f00b6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:16.656644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ef09ad8e3846668bbe3e8d314f00b6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:16.658595 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:16.722478 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ef09ad8e3846668bbe3e8d314f00b6', 'request_id': 'req-c82cd0d9-3732-4133-9e03-4f49063ec99d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.731334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ef09ad8e3846668bbe3e8d314f00b6', 'request_id': 'req-c82cd0d9-3732-4133-9e03-4f49063ec99d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.738182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ef09ad8e3846668bbe3e8d314f00b6', 'request_id': 'req-c82cd0d9-3732-4133-9e03-4f49063ec99d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.759326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] The token cache is being invalidate because project 82ef09ad8e3846668bbe3e8d314f00b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:16.759625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.760421 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c82cd0d9-3732-4133-9e03-4f49063ec99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:16.762163 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1314/2610] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:16 2022] DELETE /identity/v3/projects/82ef09ad8e3846668bbe3e8d314f00b6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:16.767646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:16.767988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:16.768349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:16.791691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:16.793708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:16.851069 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:16.870967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1297/2611] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:16.881870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781043-248a-4aa1-8895-892e0b97134b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:16.882391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781043-248a-4aa1-8895-892e0b97134b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:16.882906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781043-248a-4aa1-8895-892e0b97134b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:16.905525 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-92781043-248a-4aa1-8895-892e0b97134b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:16.906832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-92781043-248a-4aa1-8895-892e0b97134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:16.916389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1315/2612] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:16.921206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:16.933163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7e32-fab9-49bd-a7fc-048d1e45d7f0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:16.938522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:16.939338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:16.939627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:16.940028 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:16.940873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:16.941195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:16.943831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:16.958679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1298/2613] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:16.964014 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781043-248a-4aa1-8895-892e0b97134b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:16.971225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:16.979339 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781043-248a-4aa1-8895-892e0b97134b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:16.987274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:16.987972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:16.989022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:16.989999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:16.990950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:16.993294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.031240 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1316/2614] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:16 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:17.037118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.049300 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320ac84-26ef-4a8a-824e-960570acc2cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.119807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.120542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.120833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.121115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.121843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.123912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.151153 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1299/2615] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:17 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:17.155608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.169462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7e6a1-ce96-4f3b-805b-ce884d243152 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.176618 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.177319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.177608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.177885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.178645 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.180337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.193117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1317/2616] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:17.198772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.213963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b2b47-eed6-413e-a08e-3413c5e2c9d7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.219338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733368f-ac57-49e1-82f8-7360157a44bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.220007 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733368f-ac57-49e1-82f8-7360157a44bb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.220333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733368f-ac57-49e1-82f8-7360157a44bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.220608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733368f-ac57-49e1-82f8-7360157a44bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.221368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733368f-ac57-49e1-82f8-7360157a44bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.223078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733368f-ac57-49e1-82f8-7360157a44bb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.234850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1300/2617] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:36:17.240422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.252323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22073780-4384-4988-9400-8da43c64a263 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.258382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.259333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.259616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.259889 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] PATH_INFO: `/v3/projects/8f2b4e5912be4b87ad7be7a162ada75b/users/b792a2f4f35c4fdea30894f88e41e3aa/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.278869 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2b4e5912be4b87ad7be7a162ada75b, user_id=b792a2f4f35c4fdea30894f88e41e3aa, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.281919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.304864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1318/2618] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:17 2022] PUT /identity/v3/projects/8f2b4e5912be4b87ad7be7a162ada75b/users/b792a2f4f35c4fdea30894f88e41e3aa/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:17.314877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.315197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.315485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.350179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] MFA Rules not processed for user `b792a2f4f35c4fdea30894f88e41e3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:17.351523 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:17.437621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1301/2619] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:17.443028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.538362 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead6d9-a17c-4a71-9367-d0af1e4d4c4d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.543637 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.544718 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.544885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.545066 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.545815 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.548001 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.594315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1319/2620] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:17 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:17.599730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.612878 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3014ad9-1022-4d5b-bbc0-ee4ddc1257f6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.727548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:17.746207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.746718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.746923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.747196 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.747906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.749964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.779394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1302/2621] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:17 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:17.786155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.797765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ef2eb-f5fd-4b23-9f51-f8689b8900eb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.803906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.804707 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.805010 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.805304 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.806082 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.807774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.821151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1320/2622] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:17.825912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.837243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459cfd3-0b27-4c67-83e9-773d94c49c9b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.842999 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90421270-24e4-4156-a77a-879780508298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.843917 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90421270-24e4-4156-a77a-879780508298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.844236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90421270-24e4-4156-a77a-879780508298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.844508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90421270-24e4-4156-a77a-879780508298 admin admin] PATH_INFO: `/v3/projects/b9dcb8a5511b4d478a418f72615d2c86/users/c8740b8dd2914232bb88396f287c1ea2/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.865285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90421270-24e4-4156-a77a-879780508298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dcb8a5511b4d478a418f72615d2c86, user_id=c8740b8dd2914232bb88396f287c1ea2, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.868018 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90421270-24e4-4156-a77a-879780508298 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:17.885466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1303/2623] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:17 2022] PUT /identity/v3/projects/b9dcb8a5511b4d478a418f72615d2c86/users/c8740b8dd2914232bb88396f287c1ea2/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:17.890266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:17.903262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9027890-6ff0-4f6b-ad3c-a812fb93b66b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:17.988704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72de55a-e6bc-4ff5-bbc5-154d63d56da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:17.989593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72de55a-e6bc-4ff5-bbc5-154d63d56da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:17.990002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72de55a-e6bc-4ff5-bbc5-154d63d56da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:17.990405 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72de55a-e6bc-4ff5-bbc5-154d63d56da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:17.991462 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72de55a-e6bc-4ff5-bbc5-154d63d56da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:17.993521 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72de55a-e6bc-4ff5-bbc5-154d63d56da0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:18.009161 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1321/2624] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:18.015312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90421270-24e4-4156-a77a-879780508298 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:18.028414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90421270-24e4-4156-a77a-879780508298 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:18.033489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:18.034436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:18.034719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:18.035042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] PATH_INFO: `/v3/projects/b9dcb8a5511b4d478a418f72615d2c86/users/c8740b8dd2914232bb88396f287c1ea2/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:18.038577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dcb8a5511b4d478a418f72615d2c86, user_id=c8740b8dd2914232bb88396f287c1ea2, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:18.041983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:18.054242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1304/2625] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:18 2022] PUT /identity/v3/projects/b9dcb8a5511b4d478a418f72615d2c86/users/c8740b8dd2914232bb88396f287c1ea2/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:18.069842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:18.070296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:18.070620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:18.105250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] MFA Rules not processed for user `c8740b8dd2914232bb88396f287c1ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:18.107616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:18.112631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:18.202398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1322/2626] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:18.217626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:18.219750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:18.291659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d6a5-272c-4872-b013-7a236e758894 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:18.295760 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:18.297228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:18.297911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:18.298427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:18.372443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:18.374078 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:18.386095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1305/2627] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:24.331438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:24.333919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:24.352216 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c098d-f978-4154-97f4-a6d8719d7062 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:24.356193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a245764-14f7-4482-9a01-27dc659f9e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:24.356877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245764-14f7-4482-9a01-27dc659f9e2d service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:24.357158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245764-14f7-4482-9a01-27dc659f9e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:24.357439 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245764-14f7-4482-9a01-27dc659f9e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:24.369019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a245764-14f7-4482-9a01-27dc659f9e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:24.372296 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a245764-14f7-4482-9a01-27dc659f9e2d service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:24.384786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1323/2628] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:28.382501 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:34.378039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=33 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:54.270397 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:54.346321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c0c63-5521-4f4f-ace9-0674f0259e0e service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:54.351478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:54.352404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.352681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.352955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] PATH_INFO: `/v3/users/b792a2f4f35c4fdea30894f88e41e3aa` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.354259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b792a2f4f35c4fdea30894f88e41e3aa)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:54.356723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:54.421599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b792a2f4f35c4fdea30894f88e41e3aa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:54.431774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b792a2f4f35c4fdea30894f88e41e3aa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:54.443095 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b792a2f4f35c4fdea30894f88e41e3aa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:54.461609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b792a2f4f35c4fdea30894f88e41e3aa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:54.462450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b792a2f4f35c4fdea30894f88e41e3aa'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:54.477549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1306/2629] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:54 2022] DELETE /identity/v3/users/b792a2f4f35c4fdea30894f88e41e3aa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:54.484772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.485373 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.485726 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.529760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:54.531744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:54.639660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1324/2630] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:54.655691 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:54.658103 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:54.760531 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457900-c672-49f4-853d-ee0a3a3bc4e4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:54.764820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:54.765545 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.765883 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.766204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.777807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:54.779309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:54.791825 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1307/2631] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:55.167037 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:55.168878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:55.187846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572c738-9b05-4bf4-a464-496560e86361 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:55.198128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:55.199638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:55.200181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:55.200832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] PATH_INFO: `/v3/projects/8f2b4e5912be4b87ad7be7a162ada75b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:55.202956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2b4e5912be4b87ad7be7a162ada75b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:55.205827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:55.266242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2b4e5912be4b87ad7be7a162ada75b', 'request_id': 'req-b7982fb6-7ea2-41fa-bf49-942d7237da91'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.273210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2b4e5912be4b87ad7be7a162ada75b', 'request_id': 'req-b7982fb6-7ea2-41fa-bf49-942d7237da91'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.280516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2b4e5912be4b87ad7be7a162ada75b', 'request_id': 'req-b7982fb6-7ea2-41fa-bf49-942d7237da91'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.301293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] The token cache is being invalidate because project 8f2b4e5912be4b87ad7be7a162ada75b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:55.302144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.302758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.305257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1325/2632] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:55 2022] DELETE /identity/v3/projects/8f2b4e5912be4b87ad7be7a162ada75b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:55.309675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:55.321422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b66c-410d-492a-aa10-b46c104533f0 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:55.344914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:55.419750 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:55.421334 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:55.421831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:55.422336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] PATH_INFO: `/v3/users/c8740b8dd2914232bb88396f287c1ea2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:55.424906 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8740b8dd2914232bb88396f287c1ea2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:55.431104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:55.518149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8740b8dd2914232bb88396f287c1ea2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.529928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8740b8dd2914232bb88396f287c1ea2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.540794 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8740b8dd2914232bb88396f287c1ea2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.564988 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8740b8dd2914232bb88396f287c1ea2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.566914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7df92ff5-3414-48ac-b035-a3ab12a95c67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8740b8dd2914232bb88396f287c1ea2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:55.580396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1308/2633] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:55 2022] DELETE /identity/v3/users/c8740b8dd2914232bb88396f287c1ea2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:55.955283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:55.957166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:55.978891 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7982fb6-7ea2-41fa-bf49-942d7237da91 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.069840 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.070737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.071098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.071440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] PATH_INFO: `/v3/projects/b9dcb8a5511b4d478a418f72615d2c86` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.072990 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dcb8a5511b4d478a418f72615d2c86)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.075225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.122433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:56.151882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dcb8a5511b4d478a418f72615d2c86', 'request_id': 'req-6217723d-086b-4905-a18b-893fb3914ea7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:56.164518 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dcb8a5511b4d478a418f72615d2c86', 'request_id': 'req-6217723d-086b-4905-a18b-893fb3914ea7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:56.172892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dcb8a5511b4d478a418f72615d2c86', 'request_id': 'req-6217723d-086b-4905-a18b-893fb3914ea7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:56.188864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] The token cache is being invalidate because project b9dcb8a5511b4d478a418f72615d2c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:56.189158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:56.189845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6217723d-086b-4905-a18b-893fb3914ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:56.191967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1326/2634] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:55 2022] DELETE /identity/v3/projects/b9dcb8a5511b4d478a418f72615d2c86 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:56.197784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.198147 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.198490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.220798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:56.222135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:56.320446 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1309/2635] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:56.333135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.333708 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.333966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.355288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:56.356587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:56.367259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1327/2636] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:56.371898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.384186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0873f0-3d22-4598-849e-1c401de46f2d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.389614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.390383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.390713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.391056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.392351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:56.392713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.394609 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.409098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1310/2637] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:56.415370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.426760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fa071-3f0f-4aa8-a0ce-84e50585df51 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.432503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.433167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.433450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.433727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.434469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.436448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.468265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1328/2638] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:56.473805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.492599 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55d234-4fbd-47a6-b2e6-16648e47abc3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.536230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:56.583114 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.584109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.584590 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.585082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.586294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.590919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.621247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1311/2639] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:56.625844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.642335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62385-166e-41d7-a846-1d0491dae0b3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.651218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.652480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.652989 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.653486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.654968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.658303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.678067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1329/2640] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:56.682548 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.695155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35711114-da86-47e5-9e36-f1277e56f017 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.700648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fac99-bb4f-4f81-9e22-b66996a61a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.701425 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fac99-bb4f-4f81-9e22-b66996a61a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.701708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fac99-bb4f-4f81-9e22-b66996a61a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.701989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fac99-bb4f-4f81-9e22-b66996a61a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.702752 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fac99-bb4f-4f81-9e22-b66996a61a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.704507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fac99-bb4f-4f81-9e22-b66996a61a70 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.717730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1312/2641] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:56.723184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.734783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8e6f9-99ef-4a3e-90bd-540365f2cf93 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.740356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.741288 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.741573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.741851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] PATH_INFO: `/v3/projects/a6bab26d95c34a4a9fb7ca717d8cbf15/users/71f793d6a072437b8eb298abddc720f0/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.766376 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6bab26d95c34a4a9fb7ca717d8cbf15, user_id=71f793d6a072437b8eb298abddc720f0, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.769476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.792800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1330/2642] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:56 2022] PUT /identity/v3/projects/a6bab26d95c34a4a9fb7ca717d8cbf15/users/71f793d6a072437b8eb298abddc720f0/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:56.812758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.813217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.813674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.859420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] MFA Rules not processed for user `71f793d6a072437b8eb298abddc720f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:56.861403 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:56.964726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1313/2643] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:36:56.972448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.065908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579d1f-d860-4b66-b21d-25d2c225203f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.071932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.072723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.072956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.073238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.073968 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.076196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.114836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1331/2644] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:57.119451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.133518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bbf4-7bec-47c2-afb0-38d05a143196 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.203408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.204137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.204429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.204712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.205448 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.208204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.231689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1314/2645] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:36:57 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:57.236320 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.249681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c729a-150b-4892-b33a-4b57e2a990ed admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.256226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.256925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.257210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.257489 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.258236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.259885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.273340 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1332/2646] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:36:57.277898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.289812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f650f-02ec-4a59-89a1-9e07908e42f6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.295634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.296780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.297230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.297591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] PATH_INFO: `/v3/projects/15007fcd30e947b389f6a4bbb625ed82/users/e4edc069f951402cad1af8c9ccc7cf94/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.315829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=15007fcd30e947b389f6a4bbb625ed82, user_id=e4edc069f951402cad1af8c9ccc7cf94, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.318639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.336504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1315/2647] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:57 2022] PUT /identity/v3/projects/15007fcd30e947b389f6a4bbb625ed82/users/e4edc069f951402cad1af8c9ccc7cf94/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:57.341387 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.352420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd219aff-6a0c-4c2b-8a64-80cce0000814 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.381986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.441218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28d29d-94e9-4b61-8b01-823b2f772e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.442671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d29d-94e9-4b61-8b01-823b2f772e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.442958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d29d-94e9-4b61-8b01-823b2f772e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.443238 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d29d-94e9-4b61-8b01-823b2f772e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.443997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28d29d-94e9-4b61-8b01-823b2f772e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.445688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28d29d-94e9-4b61-8b01-823b2f772e18 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.458087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1333/2648] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:36:57.464903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.475453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.483537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333e01-e17b-4a8a-8d85-66d5a6df92bd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.490638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.491821 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.492213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.492529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] PATH_INFO: `/v3/projects/15007fcd30e947b389f6a4bbb625ed82/users/e4edc069f951402cad1af8c9ccc7cf94/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.494827 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15007fcd30e947b389f6a4bbb625ed82, user_id=e4edc069f951402cad1af8c9ccc7cf94, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.497956 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.513833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1316/2649] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:36:57 2022] PUT /identity/v3/projects/15007fcd30e947b389f6a4bbb625ed82/users/e4edc069f951402cad1af8c9ccc7cf94/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:57.532267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.532783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.533268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.570256 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] MFA Rules not processed for user `e4edc069f951402cad1af8c9ccc7cf94`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:57.571686 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:57.696298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1334/2650] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:57.719144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.724319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:57.853645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e84b8-55bc-4e2d-a0e5-a45b3c33f03e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.862710 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.863610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.863950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.864360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.884268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.884577 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.909496 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1317/2651] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:57.997562 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.999592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.153995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.156293 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596c0b-e365-4a33-bc11-c8bd957bc444 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.169051 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.171185 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.171860 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.172546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] PATH_INFO: `/v3/users/090960f27da74a65b8eeb82be1d31ac2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.174231 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a935-f40b-417f-aee3-9425747dcd4b service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.176375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=090960f27da74a65b8eeb82be1d31ac2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.181023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.184952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.186601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.187348 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.188122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] PATH_INFO: `/v3/users/71f793d6a072437b8eb298abddc720f0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.190642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f793d6a072437b8eb298abddc720f0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.195385 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.281085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090960f27da74a65b8eeb82be1d31ac2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.283382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.306188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090960f27da74a65b8eeb82be1d31ac2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.306525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f793d6a072437b8eb298abddc720f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.317564 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090960f27da74a65b8eeb82be1d31ac2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.318734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090960f27da74a65b8eeb82be1d31ac2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.320500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-533659a5-66ef-4569-9874-a00795e419dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090960f27da74a65b8eeb82be1d31ac2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.322272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f793d6a072437b8eb298abddc720f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.337580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f793d6a072437b8eb298abddc720f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.343056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1335/2652] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:57 2022] DELETE /identity/v3/users/090960f27da74a65b8eeb82be1d31ac2 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:58.351134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.351666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.352231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.362315 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f793d6a072437b8eb298abddc720f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.363952 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c2f41815-4e08-4356-98d1-dd14cc017ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f793d6a072437b8eb298abddc720f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:58.373386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1318/2653] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:58 2022] DELETE /identity/v3/users/71f793d6a072437b8eb298abddc720f0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:58.380012 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.380628 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.381080 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.382303 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:58.384913 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.414219 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:58.416212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.505894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1336/2654] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:58.524347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.526198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.538228 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1319/2655] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:36:58.554368 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.557173 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.569893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.613207 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2acb-7881-48e7-9a0f-86244621d55e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.617419 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.618277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.618557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.618829 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.631150 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.633024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.645679 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1337/2656] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:36:58.662648 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf143a2-78a6-49aa-ac7a-e98eb809e854 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.666891 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.668075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.668371 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.668647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.682267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.684671 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.698441 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1320/2657] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:36:59.298411 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.300314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:59.316456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ccbf1-173e-46b7-99ce-95ff144be2d6 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.322352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.323431 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.323824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.324268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] PATH_INFO: `/v3/projects/25d292c37a034d23803e395980b4219a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.325753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d292c37a034d23803e395980b4219a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.327853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.419594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.421660 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:59.444789 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d292c37a034d23803e395980b4219a', 'request_id': 'req-1ff68582-74c1-48d3-bb08-4d620cca291b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.456006 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d292c37a034d23803e395980b4219a', 'request_id': 'req-1ff68582-74c1-48d3-bb08-4d620cca291b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.472846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7300dab-0c8b-430d-afde-ed6ab66cdcaa service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.474730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d292c37a034d23803e395980b4219a', 'request_id': 'req-1ff68582-74c1-48d3-bb08-4d620cca291b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.479762 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:59.506323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] The token cache is being invalidate because project 25d292c37a034d23803e395980b4219a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:59.506807 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.508071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.516260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1338/2658] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:59 2022] DELETE /identity/v3/projects/25d292c37a034d23803e395980b4219a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:36:59.574478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.575380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.575667 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.575946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] PATH_INFO: `/v3/projects/a6bab26d95c34a4a9fb7ca717d8cbf15` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.577953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6bab26d95c34a4a9fb7ca717d8cbf15)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.580386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.662722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6bab26d95c34a4a9fb7ca717d8cbf15', 'request_id': 'req-95312df6-cc0a-49b3-8ef0-bcde25f6025f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.672646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6bab26d95c34a4a9fb7ca717d8cbf15', 'request_id': 'req-95312df6-cc0a-49b3-8ef0-bcde25f6025f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.682445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6bab26d95c34a4a9fb7ca717d8cbf15', 'request_id': 'req-95312df6-cc0a-49b3-8ef0-bcde25f6025f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.707998 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] The token cache is being invalidate because project a6bab26d95c34a4a9fb7ca717d8cbf15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:36:59.708919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.710812 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.713974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1321/2659] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:36:59 2022] DELETE /identity/v3/projects/a6bab26d95c34a4a9fb7ca717d8cbf15 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:59.719167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.725005 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:59.748155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68582-74c1-48d3-bb08-4d620cca291b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.838561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.840231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.840720 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.841215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] PATH_INFO: `/v3/users/e4edc069f951402cad1af8c9ccc7cf94` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.843608 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4edc069f951402cad1af8c9ccc7cf94)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.849208 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.950400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4edc069f951402cad1af8c9ccc7cf94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.968736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4edc069f951402cad1af8c9ccc7cf94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.979635 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4edc069f951402cad1af8c9ccc7cf94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.980516 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4edc069f951402cad1af8c9ccc7cf94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:36:59.981309 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e006baeb-5e61-413d-9e44-d6154f815ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4edc069f951402cad1af8c9ccc7cf94'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:00.010964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1339/2660] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:36:59 2022] DELETE /identity/v3/users/e4edc069f951402cad1af8c9ccc7cf94 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:00.399396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.401966 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:00.428610 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312df6-cc0a-49b3-8ef0-bcde25f6025f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:00.483396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:00.525990 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:00.527529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.528076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.528627 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] PATH_INFO: `/v3/projects/15007fcd30e947b389f6a4bbb625ed82` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.530834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] RBAC: Authorizing `identity:delete_project(project_id=15007fcd30e947b389f6a4bbb625ed82)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:00.534626 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:00.605269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15007fcd30e947b389f6a4bbb625ed82', 'request_id': 'req-ffb903bf-d3cb-4435-b960-c2931631063f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:00.615034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15007fcd30e947b389f6a4bbb625ed82', 'request_id': 'req-ffb903bf-d3cb-4435-b960-c2931631063f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:00.624310 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15007fcd30e947b389f6a4bbb625ed82', 'request_id': 'req-ffb903bf-d3cb-4435-b960-c2931631063f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:00.642329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] The token cache is being invalidate because project 15007fcd30e947b389f6a4bbb625ed82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:37:00.642935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:00.644789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ffb903bf-d3cb-4435-b960-c2931631063f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:00.652137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1322/2661] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:37:00 2022] DELETE /identity/v3/projects/15007fcd30e947b389f6a4bbb625ed82 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:00.658265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78667aff-2a2d-4f30-b794-074759511325 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.659136 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78667aff-2a2d-4f30-b794-074759511325 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.659313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78667aff-2a2d-4f30-b794-074759511325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.686515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-78667aff-2a2d-4f30-b794-074759511325 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:00.688247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-78667aff-2a2d-4f30-b794-074759511325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:00.776928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1340/2662] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:00.788002 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.788475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.788866 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.815686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:00.817903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:00.827848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1323/2663] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:00.833668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78667aff-2a2d-4f30-b794-074759511325 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.844621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78667aff-2a2d-4f30-b794-074759511325 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:00.851550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:00.852480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.852932 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.853438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.854360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:00.854801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:00.857447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:00.867535 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:00.881473 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1341/2664] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:00.884978 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.896424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c41a0-62da-4160-b4ba-684a33d6a01b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:00.901679 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:00.902544 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.902949 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.903351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.904320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:00.906358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:00.940469 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1324/2665] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:37:00 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:37:00.946445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.959438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa563dc-79ef-47e9-a55b-65ef3770a32a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.062611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.063822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.064409 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.065092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.066464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.070465 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.102832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1342/2666] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:37:00 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:37:01.107684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.120742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931881b0-3037-4492-b4f4-541eeaaa98f4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.126547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.127396 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.127723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.128041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.130023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.132075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.145898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1325/2667] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:37:01.150911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.162650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98088f6c-d622-4612-ad0c-5323dd814f1f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.167793 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32fb21-6274-435f-ad37-3b326d358231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.168764 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32fb21-6274-435f-ad37-3b326d358231 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.169158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32fb21-6274-435f-ad37-3b326d358231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.169596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32fb21-6274-435f-ad37-3b326d358231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.170702 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32fb21-6274-435f-ad37-3b326d358231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.172927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32fb21-6274-435f-ad37-3b326d358231 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.186954 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1343/2668] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:01.193467 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.211834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457838b2-1a30-4197-8b78-ee8df7b98055 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.219460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.220682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.221076 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.221445 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] PATH_INFO: `/v3/projects/2c088808610b436dab604af489e0c684/users/1321871757144cdcbb2b5239af1c09c7/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.242117 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c088808610b436dab604af489e0c684, user_id=1321871757144cdcbb2b5239af1c09c7, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.245067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.262958 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1326/2669] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:37:01 2022] PUT /identity/v3/projects/2c088808610b436dab604af489e0c684/users/1321871757144cdcbb2b5239af1c09c7/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:01.274770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.275151 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.275496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.309470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] MFA Rules not processed for user `1321871757144cdcbb2b5239af1c09c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:01.310629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:01.396322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1344/2670] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:01.410524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.412542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:01.512885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b417-ba68-4712-8b77-56bf1a23d02c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.517792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ab2c-bbe9-4494-bfd9-9c496defd2cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.518873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ab2c-bbe9-4494-bfd9-9c496defd2cf service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.519391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ab2c-bbe9-4494-bfd9-9c496defd2cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.519931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ab2c-bbe9-4494-bfd9-9c496defd2cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.531790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831ab2c-bbe9-4494-bfd9-9c496defd2cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.533447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831ab2c-bbe9-4494-bfd9-9c496defd2cf service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.545989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1327/2671] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:01.569724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.655428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a4d1d-d244-45d3-a782-9cab3f9f4af9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.661183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.662106 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.662389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.662665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] PATH_INFO: `/v3/users/1321871757144cdcbb2b5239af1c09c7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.664011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1321871757144cdcbb2b5239af1c09c7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.666004 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.672392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:01.728730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1321871757144cdcbb2b5239af1c09c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:01.747827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1321871757144cdcbb2b5239af1c09c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:01.757227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1321871757144cdcbb2b5239af1c09c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:01.758028 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1321871757144cdcbb2b5239af1c09c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:01.758690 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1321871757144cdcbb2b5239af1c09c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:01.775123 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1345/2672] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:37:01 2022] DELETE /identity/v3/users/1321871757144cdcbb2b5239af1c09c7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:01.780278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.780674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.781038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.795856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:01.805818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:01.808407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:01.911992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1328/2673] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:37:01.927856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.929759 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:02.029751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e1462-bed0-4e40-84a6-5446d674ba89 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.036896 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76d991-f6a3-4f90-a1d8-21c5e9928aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.038565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d991-f6a3-4f90-a1d8-21c5e9928aae service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.039061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d991-f6a3-4f90-a1d8-21c5e9928aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.039553 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d991-f6a3-4f90-a1d8-21c5e9928aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.052527 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76d991-f6a3-4f90-a1d8-21c5e9928aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:02.054323 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76d991-f6a3-4f90-a1d8-21c5e9928aae service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:02.069970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1346/2674] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:02.481550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.483380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:02.506955 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a05b2-7d3c-4b47-8876-3ea108e800c0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.514924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.516335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.516844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.517331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] PATH_INFO: `/v3/projects/2c088808610b436dab604af489e0c684` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.519759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c088808610b436dab604af489e0c684)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:02.524533 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:02.591291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c088808610b436dab604af489e0c684', 'request_id': 'req-37e11592-a6b0-4903-acdb-58431992af9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:02.599502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c088808610b436dab604af489e0c684', 'request_id': 'req-37e11592-a6b0-4903-acdb-58431992af9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:02.607415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c088808610b436dab604af489e0c684', 'request_id': 'req-37e11592-a6b0-4903-acdb-58431992af9b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:02.629728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] The token cache is being invalidate because project 2c088808610b436dab604af489e0c684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:37:02.630419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:02.631831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-37e11592-a6b0-4903-acdb-58431992af9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:02.635118 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1329/2675] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:37:02 2022] DELETE /identity/v3/projects/2c088808610b436dab604af489e0c684 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:02.641916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.642440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.642940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.677068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:02.679221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:02.761873 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1347/2676] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:02.771134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0870-5404-48a5-8634-38311e1251ac None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.771573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0870-5404-48a5-8634-38311e1251ac None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.771962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0870-5404-48a5-8634-38311e1251ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.798862 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-791c0870-5404-48a5-8634-38311e1251ac None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:02.802185 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-791c0870-5404-48a5-8634-38311e1251ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:02.817311 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1330/2677] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:02.822843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.835328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d767726-8b16-42b4-b323-5f5d4f57629c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.840449 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.841275 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.841561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.841833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.842579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:02.842887 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:02.844573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:02.860578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1348/2678] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:02.865900 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0870-5404-48a5-8634-38311e1251ac None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.878788 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0870-5404-48a5-8634-38311e1251ac None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.884258 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.884959 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.885244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.885525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.886246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:02.888270 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:02.921923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1331/2679] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:37:02 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:02.928822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.940437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405483f7-fd81-4d5f-9688-f5ec03dce5a1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.955674 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:03.030184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.031068 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.031346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.031693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.032656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.035684 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.078712 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1349/2680] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:37:02 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:03.087378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.100383 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad89b8-6547-4b42-a0de-9a551b8e2917 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.107497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.108453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.108859 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.109375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.110336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.112612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.131642 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1332/2681] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:03.137187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.152744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317b62a-baaa-4b33-84b5-e94725fe2bbf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.160310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31557c0b-c968-4046-876a-b1b963ba004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.161172 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31557c0b-c968-4046-876a-b1b963ba004c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.161557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31557c0b-c968-4046-876a-b1b963ba004c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.161974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31557c0b-c968-4046-876a-b1b963ba004c admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.162958 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31557c0b-c968-4046-876a-b1b963ba004c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.164966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31557c0b-c968-4046-876a-b1b963ba004c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.186435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1350/2682] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:03.194132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.208227 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540002da-e2d0-40ee-90ed-46bca47cf8b2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.216922 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.217910 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.218187 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.218475 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] PATH_INFO: `/v3/projects/aae17c9e571b42b3aec173150cf7977f/users/6073866648234fcbbdb9304d483f8e5e/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.238960 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae17c9e571b42b3aec173150cf7977f, user_id=6073866648234fcbbdb9304d483f8e5e, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.241882 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.269215 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1333/2683] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:37:03 2022] PUT /identity/v3/projects/aae17c9e571b42b3aec173150cf7977f/users/6073866648234fcbbdb9304d483f8e5e/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:03.285343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.285657 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.285988 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.335901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] MFA Rules not processed for user `6073866648234fcbbdb9304d483f8e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:03.338230 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:03.464786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1351/2684] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:03.474829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.529240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:03.585006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2082-8a05-4df3-a8f9-1c6b53d756c0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.591017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.591766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.592085 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.592369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.593104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.595225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.640792 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1334/2685] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:37:03 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:37:03.645864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.661348 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22040d6b-4a75-41d0-880d-af440a2b8293 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.765450 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.766479 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.766640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.767098 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.768022 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.770444 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.806058 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1352/2686] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:37:03 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:03.811894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.827601 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddce2a8-847d-4f1a-9910-d91a63d1479b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.833841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.834550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.834839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.835120 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.835918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.837786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.854549 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1335/2687] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:37:03.859472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.872332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cfe9b-4b3b-47b6-97a0-e02963d0e368 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.881094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248813c-ac6a-4e28-9768-13bb35bf26d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.881826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248813c-ac6a-4e28-9768-13bb35bf26d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.882086 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248813c-ac6a-4e28-9768-13bb35bf26d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.882363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248813c-ac6a-4e28-9768-13bb35bf26d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.883333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248813c-ac6a-4e28-9768-13bb35bf26d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.886197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248813c-ac6a-4e28-9768-13bb35bf26d2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.903551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1353/2688] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:03.909323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.925074 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae53a39-2c5e-4993-b1dd-39d4bae200d6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.937357 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.939192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.939703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.940333 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] PATH_INFO: `/v3/projects/1c4b250ebc8443bf9a91dfdead7ab0f7/users/772317d6d2d248448cb04288c9a16685/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.973914 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4b250ebc8443bf9a91dfdead7ab0f7, user_id=772317d6d2d248448cb04288c9a16685, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.976790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.996929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1336/2689] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:37:03 2022] PUT /identity/v3/projects/1c4b250ebc8443bf9a91dfdead7ab0f7/users/772317d6d2d248448cb04288c9a16685/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:04.010725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.011190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.011446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.018507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.049461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] MFA Rules not processed for user `772317d6d2d248448cb04288c9a16685`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:04.051836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:04.170964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1354/2690] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:04.177895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.271414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f232c7-b0f6-4950-9775-af7000f54fb2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.277842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.278402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.278816 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.279107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.280124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.283699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.331309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1337/2691] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:04.337491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.353836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a60cb-7193-4581-9e5a-0458adc69ec0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.454786 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.455728 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.456175 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.456590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.457533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.459916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.492121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1355/2692] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:37:04 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:04.498865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.513179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b6dc2-ad7d-450f-b0e3-26f18276c51e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.523400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.524038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.524565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.524840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.525873 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.528898 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.543042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1338/2693] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:04.549203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.562280 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b080b2-c139-4804-82f0-58ce063f0d6e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.568499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.569665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.570007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.570357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] PATH_INFO: `/v3/projects/29e93229660b47759b58e4c5c9f44e8b/users/c81502f2ca7b40d09c590c586b78b4dd/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.592463 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e93229660b47759b58e4c5c9f44e8b, user_id=c81502f2ca7b40d09c590c586b78b4dd, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.595885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.617738 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1356/2694] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:37:04 2022] PUT /identity/v3/projects/29e93229660b47759b58e4c5c9f44e8b/users/c81502f2ca7b40d09c590c586b78b4dd/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:04.622463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.634742 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02c8b3-0936-4310-b1bd-5cea18f60add admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.695162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.710136 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e9ae4-ca1f-4fb4-804a-a09a8256e43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.710843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e9ae4-ca1f-4fb4-804a-a09a8256e43c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.711126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e9ae4-ca1f-4fb4-804a-a09a8256e43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.711400 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e9ae4-ca1f-4fb4-804a-a09a8256e43c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.712213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e9ae4-ca1f-4fb4-804a-a09a8256e43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.713885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e9ae4-ca1f-4fb4-804a-a09a8256e43c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.726253 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1339/2695] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:04.731675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.744344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2a9-9c61-4026-9f40-a9b22f69ca32 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.749382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.750289 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.750570 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.750843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] PATH_INFO: `/v3/projects/29e93229660b47759b58e4c5c9f44e8b/users/c81502f2ca7b40d09c590c586b78b4dd/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.753190 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e93229660b47759b58e4c5c9f44e8b, user_id=c81502f2ca7b40d09c590c586b78b4dd, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.757181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.770016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1357/2696] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:37:04 2022] PUT /identity/v3/projects/29e93229660b47759b58e4c5c9f44e8b/users/c81502f2ca7b40d09c590c586b78b4dd/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:04.781309 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.781551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.782503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.817477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] MFA Rules not processed for user `c81502f2ca7b40d09c590c586b78b4dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:04.819065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:04.916030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1340/2697] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:37:04.933661 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.935529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:04.998579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.014061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b443c-6dce-458f-9950-e98f9e3a92c8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.018363 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.019833 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.020268 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.020630 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.108752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.110550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.122734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1358/2698] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:37:06.819975 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.822686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:06.844885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8943584-16df-4972-96d1-b65928e9e403 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.850192 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.850943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.851290 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.851652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.966665 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.969008 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.984850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1341/2699] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:07.766900 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.781146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033333d-e3fc-4d1d-870b-bf4c68ca1b50 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.786468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0155326-9abd-4499-bf8a-5f6d8e488e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.787155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0155326-9abd-4499-bf8a-5f6d8e488e68 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.787445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0155326-9abd-4499-bf8a-5f6d8e488e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.787723 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0155326-9abd-4499-bf8a-5f6d8e488e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.799120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0155326-9abd-4499-bf8a-5f6d8e488e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.801488 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0155326-9abd-4499-bf8a-5f6d8e488e68 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.812864 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1359/2700] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:37:12.072507 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.188381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c526dd-97b1-4dc5-b93c-e02f2db66e06 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.199132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.200824 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.201305 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.201756 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] PATH_INFO: `/v3/users/6073866648234fcbbdb9304d483f8e5e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.204973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6073866648234fcbbdb9304d483f8e5e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.211450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.246697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:12.309885 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6073866648234fcbbdb9304d483f8e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:12.320244 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6073866648234fcbbdb9304d483f8e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:12.338909 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6073866648234fcbbdb9304d483f8e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:12.358804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6073866648234fcbbdb9304d483f8e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:12.359573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6073866648234fcbbdb9304d483f8e5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:12.368436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1342/2701] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:37:12 2022] DELETE /identity/v3/users/6073866648234fcbbdb9304d483f8e5e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:12.376234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.376918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.377221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.402217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:12.403572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:12.502976 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1360/2702] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:12.521149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.524101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:12.650042 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0567e-660e-43fa-9bd5-60564d6fbf6b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.655246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.656220 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.656538 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.656972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.671099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.673412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.686603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1343/2703] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:13.293266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.295094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:13.313310 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195296e-8114-4a5d-afb9-ffe0ba9229c5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.318844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.320671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.320671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.320671 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] PATH_INFO: `/v3/projects/aae17c9e571b42b3aec173150cf7977f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.322467 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae17c9e571b42b3aec173150cf7977f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:13.326374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:13.440214 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae17c9e571b42b3aec173150cf7977f', 'request_id': 'req-3a8aa9fe-6365-45e7-b9ca-270b662088b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.450576 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae17c9e571b42b3aec173150cf7977f', 'request_id': 'req-3a8aa9fe-6365-45e7-b9ca-270b662088b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.459960 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae17c9e571b42b3aec173150cf7977f', 'request_id': 'req-3a8aa9fe-6365-45e7-b9ca-270b662088b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.480219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] The token cache is being invalidate because project aae17c9e571b42b3aec173150cf7977f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:37:13.480620 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.481389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.483294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1361/2704] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:37:13 2022] DELETE /identity/v3/projects/aae17c9e571b42b3aec173150cf7977f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:13.490288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.501613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2541a5-18f1-449d-9d5c-b81b271aa466 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.596505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.598104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.598608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.599087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] PATH_INFO: `/v3/users/772317d6d2d248448cb04288c9a16685` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.601437 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=772317d6d2d248448cb04288c9a16685)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:13.605236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:13.677647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:13.694905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '772317d6d2d248448cb04288c9a16685'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.706641 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '772317d6d2d248448cb04288c9a16685'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.718113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '772317d6d2d248448cb04288c9a16685'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.736274 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '772317d6d2d248448cb04288c9a16685'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.738102 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '772317d6d2d248448cb04288c9a16685'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:13.746388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1344/2705] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:37:13 2022] DELETE /identity/v3/users/772317d6d2d248448cb04288c9a16685 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:14.176388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.178971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:14.207950 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8aa9fe-6365-45e7-b9ca-270b662088b4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.217094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.321436 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.322986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.323531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.324008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] PATH_INFO: `/v3/projects/1c4b250ebc8443bf9a91dfdead7ab0f7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.326346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4b250ebc8443bf9a91dfdead7ab0f7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.330177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.424725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4b250ebc8443bf9a91dfdead7ab0f7', 'request_id': 'req-919f19cb-1143-4918-ae0f-5cb621a390ee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.436225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4b250ebc8443bf9a91dfdead7ab0f7', 'request_id': 'req-919f19cb-1143-4918-ae0f-5cb621a390ee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.445372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4b250ebc8443bf9a91dfdead7ab0f7', 'request_id': 'req-919f19cb-1143-4918-ae0f-5cb621a390ee'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.468561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] The token cache is being invalidate because project 1c4b250ebc8443bf9a91dfdead7ab0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:37:14.469079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.470160 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.473374 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1362/2706] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:37:14 2022] DELETE /identity/v3/projects/1c4b250ebc8443bf9a91dfdead7ab0f7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:14.479466 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.481879 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:14.501359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149532d2-cce0-4ddd-b1fd-679437e3b117 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.583979 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.584923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.585207 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.585479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] PATH_INFO: `/v3/users/c81502f2ca7b40d09c590c586b78b4dd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.586892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81502f2ca7b40d09c590c586b78b4dd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.589370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.653719 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81502f2ca7b40d09c590c586b78b4dd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.664844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81502f2ca7b40d09c590c586b78b4dd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.679892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81502f2ca7b40d09c590c586b78b4dd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.701057 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81502f2ca7b40d09c590c586b78b4dd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.702288 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3deecd8d-d357-4d20-9183-0eba519192c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81502f2ca7b40d09c590c586b78b4dd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:14.710312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1345/2707] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:37:14 2022] DELETE /identity/v3/users/c81502f2ca7b40d09c590c586b78b4dd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:15.080914 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.083184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:15.098056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f19cb-1143-4918-ae0f-5cb621a390ee admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.171468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.172734 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.173030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.173291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] PATH_INFO: `/v3/projects/29e93229660b47759b58e4c5c9f44e8b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.175292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e93229660b47759b58e4c5c9f44e8b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.177812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.232285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:15.243878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e93229660b47759b58e4c5c9f44e8b', 'request_id': 'req-2360b049-973c-4ee3-87a4-b813a8920bcb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:15.252857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e93229660b47759b58e4c5c9f44e8b', 'request_id': 'req-2360b049-973c-4ee3-87a4-b813a8920bcb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:15.260534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e93229660b47759b58e4c5c9f44e8b', 'request_id': 'req-2360b049-973c-4ee3-87a4-b813a8920bcb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:15.281198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] The token cache is being invalidate because project 29e93229660b47759b58e4c5c9f44e8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:37:15.281706 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:15.283044 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2360b049-973c-4ee3-87a4-b813a8920bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:37:15.285640 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1363/2708] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:37:15 2022] DELETE /identity/v3/projects/29e93229660b47759b58e4c5c9f44e8b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:15.291758 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.292098 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.292388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.315009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:15.317842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:15.360198 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:15.404432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1346/2709] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:37:15.413826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.416105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.416105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.434902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:15.436282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:15.445209 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1364/2710] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:15.450062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.462045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dbe16-0aea-4364-a012-8fc39c5d60b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.467618 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.469783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.470293 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.470789 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.472810 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:15.473381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.476576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.496542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1347/2711] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:15.504206 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.541892 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837a49b-12ae-45b9-8cb3-33bd83a1cfc2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.549666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.551198 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.551681 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.552215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.553386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.555732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.595696 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1365/2712] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:37:15 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:15.602890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.622689 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd81e6-3468-4a98-ac92-fe405dba9998 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.730920 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.731629 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.731951 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.732320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.733131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.735817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.775733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1348/2713] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:37:15 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:15.780091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.794266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329adaec-1b47-42d7-82c8-18629281dd71 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.803752 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.805453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.806228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.806424 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.808856 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.812328 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.813478 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.816667 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:15.842539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1366/2714] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:37:15.850534 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.868135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0920ca-6037-41ef-9d79-b75f664f6b9f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.876204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.877577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.878938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.878938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.880832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.882806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.903166 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1367/2715] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:37:15.910265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.922352 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5faa7d-8963-46b9-baaa-d6d0d8cb3a0c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.930670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf860b-8a10-4ebd-9e4d-11dadfb957dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.931600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf860b-8a10-4ebd-9e4d-11dadfb957dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.931920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf860b-8a10-4ebd-9e4d-11dadfb957dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.932324 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf860b-8a10-4ebd-9e4d-11dadfb957dd admin admin] PATH_INFO: `/v3/projects/4283712d27ef4a748657f3bf788ba2c7/users/5ffb4685481844bc871c5b2607b5aadb/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.947479 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0781cb0-38fb-496b-b803-9bafcb92a3c2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.954737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.957031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.957445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf860b-8a10-4ebd-9e4d-11dadfb957dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4283712d27ef4a748657f3bf788ba2c7, user_id=5ffb4685481844bc871c5b2607b5aadb, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.957674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.958312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.960796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:15.961538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf860b-8a10-4ebd-9e4d-11dadfb957dd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.977894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.979553 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.981167 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1368/2716] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:37:15 2022] PUT /identity/v3/projects/4283712d27ef4a748657f3bf788ba2c7/users/5ffb4685481844bc871c5b2607b5aadb/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:15.990701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34a3e-f31c-4787-9ded-c12442d58a42 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.991184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34a3e-f31c-4787-9ded-c12442d58a42 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.991587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34a3e-f31c-4787-9ded-c12442d58a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.998546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1349/2717] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:37:16.018346 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d1d34a3e-f31c-4787-9ded-c12442d58a42 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:16.021458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d1d34a3e-f31c-4787-9ded-c12442d58a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:16.124573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1369/2718] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:37:16.141275 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.142857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:16.219321 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:16.250287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1eaa54-8ae5-447b-a510-2612dfb17e3c service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.332484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:16.333251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.333695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.333846 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.349213 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:16.351625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:16.365398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1350/2719] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:37:17.338398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63958783-249a-41a7-b29f-1d23cbc2a2e3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.338398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63958783-249a-41a7-b29f-1d23cbc2a2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.338398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63958783-249a-41a7-b29f-1d23cbc2a2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.388455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-63958783-249a-41a7-b29f-1d23cbc2a2e3 None None] MFA Rules not processed for user `5ffb4685481844bc871c5b2607b5aadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:17.390639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-63958783-249a-41a7-b29f-1d23cbc2a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.431122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.433566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.447927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9f026-c094-464f-81a0-159dfa85e0d1 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.453025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.453817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.454166 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.454484 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.458453 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.473194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.475399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.476583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:17.492919 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1351/2720] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:17.506548 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1370/2721] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:17.532405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.535546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.639414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae6748-fa00-4cb2-bade-645407b5ea63 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.639880 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae6748-fa00-4cb2-bade-645407b5ea63 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.640407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae6748-fa00-4cb2-bade-645407b5ea63 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.643219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1371/2722] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:37:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:17.653133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2228b-7d2f-466a-85b8-b3abece0d668 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.653585 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2228b-7d2f-466a-85b8-b3abece0d668 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.654061 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2228b-7d2f-466a-85b8-b3abece0d668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.660727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679edc7-d047-40d0-ae62-d3ad5a650372 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.666406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.667122 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.667407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.667687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.680918 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.683141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.685816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2bd2228b-7d2f-466a-85b8-b3abece0d668 None None] MFA Rules not processed for user `558f4050abe8467aa527d73a49f101b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:17.687217 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2bd2228b-7d2f-466a-85b8-b3abece0d668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.696524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1352/2723] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:17.699586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1372/2724] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:17.722402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.733851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e807f82-7d1e-4277-a7ea-0e625de79a03 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.740405 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d653b7-2169-4202-97b1-fb916ddbef9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.741226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d653b7-2169-4202-97b1-fb916ddbef9f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.741585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d653b7-2169-4202-97b1-fb916ddbef9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.741946 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d653b7-2169-4202-97b1-fb916ddbef9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.752320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d653b7-2169-4202-97b1-fb916ddbef9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.754884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d653b7-2169-4202-97b1-fb916ddbef9f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.767024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1353/2725] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:37:17.885907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716607f-acf2-4a08-addf-9a5a7b3813a0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.886194 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716607f-acf2-4a08-addf-9a5a7b3813a0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.886484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716607f-acf2-4a08-addf-9a5a7b3813a0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.888729 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1373/2726] 10.222.0.30 () {54 vars in 921 bytes} [Thu Jun 16 18:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:17.900947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd8e62-7be0-4ecb-84ac-68f6464d3f06 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.901451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd8e62-7be0-4ecb-84ac-68f6464d3f06 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.901941 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd8e62-7be0-4ecb-84ac-68f6464d3f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.930318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9ecd8e62-7be0-4ecb-84ac-68f6464d3f06 None None] MFA Rules not processed for user `793065092530407983e5da2446d2e939`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:17.931856 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9ecd8e62-7be0-4ecb-84ac-68f6464d3f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:18.080351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1354/2727] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Jun 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:18.101129 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716607f-acf2-4a08-addf-9a5a7b3813a0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.116956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716607f-acf2-4a08-addf-9a5a7b3813a0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.125300 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95992620-b65e-4219-af55-1891c00d2a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.126265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992620-b65e-4219-af55-1891c00d2a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.126749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992620-b65e-4219-af55-1891c00d2a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.127197 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992620-b65e-4219-af55-1891c00d2a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.156475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95992620-b65e-4219-af55-1891c00d2a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.162650 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95992620-b65e-4219-af55-1891c00d2a35 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.190893 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1374/2728] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:37:19.472893 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e240-6004-46e3-a890-f5c76898dab8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.474436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e240-6004-46e3-a890-f5c76898dab8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.474970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e240-6004-46e3-a890-f5c76898dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.487297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:19.507037 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4bc1e240-6004-46e3-a890-f5c76898dab8 None None] MFA Rules not processed for user `5ffb4685481844bc871c5b2607b5aadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:19.509612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4bc1e240-6004-46e3-a890-f5c76898dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:19.518272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1355/2729] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:37:28.177993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=16 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.498463 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.183213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.183969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.184443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.216012 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] MFA Rules not processed for user `5ffb4685481844bc871c5b2607b5aadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:38.218261 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:38.229319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1375/2730] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:37:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:37:48.206962 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.015429 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdb3ca-12ba-4070-9566-41dfb0411049 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.016331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdb3ca-12ba-4070-9566-41dfb0411049 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.016331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdb3ca-12ba-4070-9566-41dfb0411049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.040817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1bfdb3ca-12ba-4070-9566-41dfb0411049 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:00.042109 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1bfdb3ca-12ba-4070-9566-41dfb0411049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:00.116624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1356/2731] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Jun 16 18:38:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:38:00.177243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.179547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:00.279378 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dcef2-54c9-4cdb-89d3-c1d8a974e99f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.285771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88cc77-dd21-43e7-9fd6-d7d3d8a9294c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.286490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88cc77-dd21-43e7-9fd6-d7d3d8a9294c service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.286835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88cc77-dd21-43e7-9fd6-d7d3d8a9294c service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.288455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88cc77-dd21-43e7-9fd6-d7d3d8a9294c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.299882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88cc77-dd21-43e7-9fd6-d7d3d8a9294c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:00.302285 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88cc77-dd21-43e7-9fd6-d7d3d8a9294c service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:00.316253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1376/2732] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:38:10.106654 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=25 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:10.310648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=32 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:11.066904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.067676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.068384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.111239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] MFA Rules not processed for user `ebdf8e3cd1b44dcba37cdcfb4af7db2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:11.114320 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:11.270685 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1357/2733] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:38:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:38:19.427711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.428631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.428631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.457778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] MFA Rules not processed for user `93c4664744bf4bfbbc5436c5866fcb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:19.459964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:19.468802 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1377/2734] 10.222.0.30 () {58 vars in 967 bytes} [Thu Jun 16 18:38:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:38:19.485976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.500182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0cdc3-d65e-4d19-aace-9c890a1cfd44 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.506369 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5081e9bf640679c72beb1b18f3e78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.507058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.507370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.507652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.521925 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.523908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.534912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1358/2735] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:38:21.783257 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.797297 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859f96c-0aa2-4241-9eea-d03c2a79d417 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.804626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:21.805551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.806113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.806356 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] PATH_INFO: `/v3/users/5ffb4685481844bc871c5b2607b5aadb` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.807774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ffb4685481844bc871c5b2607b5aadb)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:21.809852 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.889884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ffb4685481844bc871c5b2607b5aadb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:21.904196 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ffb4685481844bc871c5b2607b5aadb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:21.912360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ffb4685481844bc871c5b2607b5aadb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:21.913165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ffb4685481844bc871c5b2607b5aadb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:21.913881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e817a778-447c-4cda-a52f-c1083a283353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ffb4685481844bc871c5b2607b5aadb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:21.929713 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1378/2736] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:38:21 2022] DELETE /identity/v3/users/5ffb4685481844bc871c5b2607b5aadb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:22.138180 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.155412 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731004c-bdcf-4314-9a78-09f0ec7d137d service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.239047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.240614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.241212 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.241728 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] PATH_INFO: `/v3/projects/4283712d27ef4a748657f3bf788ba2c7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.244204 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4283712d27ef4a748657f3bf788ba2c7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.247980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.309565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4283712d27ef4a748657f3bf788ba2c7', 'request_id': 'req-72c533bc-bd66-4cb1-aaba-7d71dd41c695'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:22.317017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4283712d27ef4a748657f3bf788ba2c7', 'request_id': 'req-72c533bc-bd66-4cb1-aaba-7d71dd41c695'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:22.324570 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4283712d27ef4a748657f3bf788ba2c7', 'request_id': 'req-72c533bc-bd66-4cb1-aaba-7d71dd41c695'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:22.349235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] The token cache is being invalidate because project 4283712d27ef4a748657f3bf788ba2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:38:22.349887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:22.350683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-72c533bc-bd66-4cb1-aaba-7d71dd41c695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:22.352358 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1359/2737] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:38:22 2022] DELETE /identity/v3/projects/4283712d27ef4a748657f3bf788ba2c7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:22.357742 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.357992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.358203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.380648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:22.381977 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:22.458717 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1379/2738] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:38:22.472458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.472976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.473487 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.504224 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:22.506005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:22.516170 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1360/2739] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:38:22.521883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.538107 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dee77-9d58-4d4e-a0c4-eb53a803460e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.546116 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.547382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.548400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.549182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.550313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.550753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.553147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.576714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1380/2740] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.582380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.592551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237f224-983e-4eba-88d3-65f4c66422e2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.598947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.599639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.599926 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.600230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.600984 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.603398 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.624303 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.654915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1361/2741] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:38:22 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:22.659783 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.676415 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f9147-3a01-4f31-b6a1-400954503c81 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.739959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.754533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.755291 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.755577 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.755881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.756653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.758638 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.791458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1381/2742] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:38:22 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:22.794084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.807113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21572638-13a3-4f01-b3db-ef742191a7bd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.812477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.813894 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.814188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.814455 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.815281 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.816876 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.831804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1362/2743] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:38:22.837011 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.856245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92d606-0319-4222-80a7-05061bbb6c0f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.863135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264c52b-0b09-40d6-85cc-b847bfde99d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.863894 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264c52b-0b09-40d6-85cc-b847bfde99d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.864556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264c52b-0b09-40d6-85cc-b847bfde99d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.864902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264c52b-0b09-40d6-85cc-b847bfde99d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.865823 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264c52b-0b09-40d6-85cc-b847bfde99d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.868654 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264c52b-0b09-40d6-85cc-b847bfde99d5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.885964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1382/2744] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:38:22.889404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.908107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b118ce-8452-4215-a5ef-7284425bdbeb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.912996 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.914101 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.914498 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.914967 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] PATH_INFO: `/v3/projects/c2d1a54f1fd64c41a8e88ab5512a1f22/users/3121b08a65cf4f2fb6105e1f570fc56f/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.938056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d1a54f1fd64c41a8e88ab5512a1f22, user_id=3121b08a65cf4f2fb6105e1f570fc56f, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.941277 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.961722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1363/2745] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:38:22 2022] PUT /identity/v3/projects/c2d1a54f1fd64c41a8e88ab5512a1f22/users/3121b08a65cf4f2fb6105e1f570fc56f/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:22.974336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384c850-9437-4013-902d-1f56e696949c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.974832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384c850-9437-4013-902d-1f56e696949c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.975393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384c850-9437-4013-902d-1f56e696949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.027091 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3384c850-9437-4013-902d-1f56e696949c None None] MFA Rules not processed for user `3121b08a65cf4f2fb6105e1f570fc56f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:23.031113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3384c850-9437-4013-902d-1f56e696949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:23.131110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1383/2746] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:38:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:38:23.145045 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:23.147162 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:23.240776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773f56-7526-4b7c-9f47-d9b1f4dadfab admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:23.244447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2948b-aab6-4080-92af-cd0ef1891b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:23.245100 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2948b-aab6-4080-92af-cd0ef1891b65 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.245381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2948b-aab6-4080-92af-cd0ef1891b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.245655 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2948b-aab6-4080-92af-cd0ef1891b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.254341 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2948b-aab6-4080-92af-cd0ef1891b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:23.255796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2948b-aab6-4080-92af-cd0ef1891b65 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:23.264813 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1364/2747] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:38:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:38:25.914445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384c850-9437-4013-902d-1f56e696949c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:26.026368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384c850-9437-4013-902d-1f56e696949c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.033252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.035087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.035629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.036704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] PATH_INFO: `/v3/users/3121b08a65cf4f2fb6105e1f570fc56f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.039243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3121b08a65cf4f2fb6105e1f570fc56f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.043118 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.106872 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3121b08a65cf4f2fb6105e1f570fc56f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:26.120832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:26.122491 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3121b08a65cf4f2fb6105e1f570fc56f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:26.133189 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3121b08a65cf4f2fb6105e1f570fc56f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:26.134517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3121b08a65cf4f2fb6105e1f570fc56f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:26.135651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3121b08a65cf4f2fb6105e1f570fc56f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:26.152084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1384/2748] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:38:25 2022] DELETE /identity/v3/users/3121b08a65cf4f2fb6105e1f570fc56f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:26.156875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.157525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.157735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.178392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:26.179801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:26.268623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1365/2749] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:38:26.285906 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:26.287482 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:26.381573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb6e35-7e9d-4799-acff-55a8137c4ea6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.385660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.386375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.386665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.386941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.398269 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.399781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.412396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1385/2750] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:38:26.950783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:26.952540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:26.975115 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cf6a0-11f3-4cf0-bdbb-9d7d1b05512a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.982581 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.984210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.984731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.985863 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] PATH_INFO: `/v3/projects/c2d1a54f1fd64c41a8e88ab5512a1f22` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.989233 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d1a54f1fd64c41a8e88ab5512a1f22)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.994174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:27.059807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d1a54f1fd64c41a8e88ab5512a1f22', 'request_id': 'req-ec6421e2-2df5-4a68-b4c2-05355e456797'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:27.067265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:27.069151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d1a54f1fd64c41a8e88ab5512a1f22', 'request_id': 'req-ec6421e2-2df5-4a68-b4c2-05355e456797'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:27.077804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d1a54f1fd64c41a8e88ab5512a1f22', 'request_id': 'req-ec6421e2-2df5-4a68-b4c2-05355e456797'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:27.099939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] The token cache is being invalidate because project c2d1a54f1fd64c41a8e88ab5512a1f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:38:27.100510 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:27.101616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:38:27.104571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1366/2751] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:38:26 2022] DELETE /identity/v3/projects/c2d1a54f1fd64c41a8e88ab5512a1f22 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:36.407414 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=33,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.092550 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.288938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.291221 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:59.314026 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6a26f-fdcb-4515-a429-9603db7f7101 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.398060 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.398983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.399380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.399761 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.402204 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:59.514157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:59.516727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:59.533250 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1386/2752] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:38:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:00.833047 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.835347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:00.865294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6421e2-2df5-4a68-b4c2-05355e456797 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.873485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.874596 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.875972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.876442 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.881058 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:00.906936 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.909822 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.933530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1367/2753] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:39:03.793319 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:03.794978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:03.808550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad3c80-0e0f-4d95-b88d-a54547a4364c service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:03.813572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:03.814509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:03.814898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:03.815276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:03.817597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:03.916651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:03.919509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:03.940318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1387/2754] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:39:08.082071 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:08.087026 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:08.227029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9b09-02f6-4177-9f2d-f4c39fbbd2e4 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:08.232841 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:08.233616 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:08.234000 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:08.234336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:08.236624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:08.249735 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:08.251542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:08.263864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1368/2755] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:12.558234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:12.563966 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:12.578345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702061e-fe02-483e-9f69-1ff1e567c5c3 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:12.583546 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:12.584859 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:12.585148 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:12.585550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] PATH_INFO: `/v3/users/0cc31f4b39704173858410ae52ac2c66` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:12.587074 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc31f4b39704173858410ae52ac2c66)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:12.589144 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:12.644002 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:12.672594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc31f4b39704173858410ae52ac2c66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:12.694749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc31f4b39704173858410ae52ac2c66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:12.705034 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc31f4b39704173858410ae52ac2c66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:12.706117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc31f4b39704173858410ae52ac2c66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:12.706975 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc31f4b39704173858410ae52ac2c66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:12.728486 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1388/2756] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:12 2022] DELETE /identity/v3/users/0cc31f4b39704173858410ae52ac2c66 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:39:13.069968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:13.087414 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effb9b8-0d85-4ac2-9551-e7760e1ee811 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:13.180075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:13.181727 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:13.182256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:13.182771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] PATH_INFO: `/v3/projects/4d9f254f568e4bd98b2e4087c3a32c04` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:13.184897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9f254f568e4bd98b2e4087c3a32c04)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:13.188384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:13.264621 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9f254f568e4bd98b2e4087c3a32c04', 'request_id': 'req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.275757 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9f254f568e4bd98b2e4087c3a32c04', 'request_id': 'req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.288565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9f254f568e4bd98b2e4087c3a32c04', 'request_id': 'req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.307242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] The token cache is being invalidate because project 4d9f254f568e4bd98b2e4087c3a32c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:13.307731 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.308889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.310904 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1369/2757] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:13 2022] DELETE /identity/v3/projects/4d9f254f568e4bd98b2e4087c3a32c04 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:13.316792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:13.318830 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:13.334941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50e37d-67d4-49ea-a56c-75532ac37c3a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:13.412264 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:13.414072 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:13.414593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:13.415059 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] PATH_INFO: `/v3/users/2deb86b8eef34c2d934d63f5ead001b4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:13.417420 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2deb86b8eef34c2d934d63f5ead001b4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:13.421812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:13.491000 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2deb86b8eef34c2d934d63f5ead001b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.508573 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2deb86b8eef34c2d934d63f5ead001b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.518886 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2deb86b8eef34c2d934d63f5ead001b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.519724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2deb86b8eef34c2d934d63f5ead001b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.520440 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-244f0dfb-35b0-459e-bc72-fd95d91ded51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2deb86b8eef34c2d934d63f5ead001b4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.537195 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1389/2758] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:13 2022] DELETE /identity/v3/users/2deb86b8eef34c2d934d63f5ead001b4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:13.759486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:13.762748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:13.789500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6715ac-1c2b-4d0a-9a79-aebb8117f298 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:13.796940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:13.879239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:13.880815 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:13.881346 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:13.881855 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] PATH_INFO: `/v3/projects/f955e3c1c6ed4b99b5ab4009d470a806` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:13.884167 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f955e3c1c6ed4b99b5ab4009d470a806)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:13.887890 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:13.957850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f955e3c1c6ed4b99b5ab4009d470a806', 'request_id': 'req-9efbb792-1285-495e-be0d-6567de3f076d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.964529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f955e3c1c6ed4b99b5ab4009d470a806', 'request_id': 'req-9efbb792-1285-495e-be0d-6567de3f076d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.973521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f955e3c1c6ed4b99b5ab4009d470a806', 'request_id': 'req-9efbb792-1285-495e-be0d-6567de3f076d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.989573 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] The token cache is being invalidate because project f955e3c1c6ed4b99b5ab4009d470a806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:13.989994 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.990746 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9efbb792-1285-495e-be0d-6567de3f076d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:13.992759 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1370/2759] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:13 2022] DELETE /identity/v3/projects/f955e3c1c6ed4b99b5ab4009d470a806 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:13.998292 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:13.998920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:13.998920 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.021434 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:14.022817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:14.049846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:14.101135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1390/2760] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:14.111106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.111716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.112091 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.144974 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:14.146268 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:14.154905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1371/2761] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:14.160330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.171501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf13e09-e7ce-4606-9995-f9be9d2d2cf0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.176959 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.177778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.178131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.178483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.179483 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:14.179868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.181745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.195113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1391/2762] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:14.199929 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.213075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6605e-5ddd-4a34-bcf2-d563cb4784e7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.218995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.219652 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.219942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.220349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.221082 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.223044 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.261530 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1372/2763] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:14 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:39:14.263839 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.282743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a86d0-27ac-4055-94b2-41c8be3adaf3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.355318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.356265 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.356633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.356953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.357700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.360433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.387442 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1392/2764] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:14 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:14.391674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.404029 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca2da3-9667-42bb-be75-c65fe9ff73ca admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.409304 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.410003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.410283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.410556 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.411313 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.412992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.427053 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1373/2765] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:14.431389 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.444532 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640a43e-6587-4d9d-b32b-0920eb379526 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.449883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09d74-1879-4670-83ad-2c6dff69a880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.450653 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09d74-1879-4670-83ad-2c6dff69a880 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.451205 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09d74-1879-4670-83ad-2c6dff69a880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.451550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09d74-1879-4670-83ad-2c6dff69a880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.453008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea09d74-1879-4670-83ad-2c6dff69a880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.455085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea09d74-1879-4670-83ad-2c6dff69a880 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.466464 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1393/2766] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:39:14.471730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.482734 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec786-6452-45c4-94ac-4dfbedd22ec9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.488436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.489362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.489650 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.489933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] PATH_INFO: `/v3/projects/dff4784f6ab3499b88cb38c81780b754/users/e5cbf8d443de4a9fb627ef316fbd033f/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.511895 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff4784f6ab3499b88cb38c81780b754, user_id=e5cbf8d443de4a9fb627ef316fbd033f, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.515223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.532714 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1374/2767] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:14 2022] PUT /identity/v3/projects/dff4784f6ab3499b88cb38c81780b754/users/e5cbf8d443de4a9fb627ef316fbd033f/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:14.538980 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fea5f8-0ea5-4127-8240-18c3e103cf04 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.539215 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fea5f8-0ea5-4127-8240-18c3e103cf04 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.539517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fea5f8-0ea5-4127-8240-18c3e103cf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.561746 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-12fea5f8-0ea5-4127-8240-18c3e103cf04 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:14.563870 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-12fea5f8-0ea5-4127-8240-18c3e103cf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:14.641243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1394/2768] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:14.652638 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:14.654795 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:14.698165 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:14.729518 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d2f48-1e8a-443a-8a76-9f69b0564fb9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:14.734528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:14.735230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.735777 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.736119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.747189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:14.748911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:14.760804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1375/2769] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:39:20.445093 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.445667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.446089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.507202 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] MFA Rules not processed for user `e5cbf8d443de4a9fb627ef316fbd033f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:20.509298 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:20.541360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:20.619084 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1395/2770] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:20.626796 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:20.658056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913675-0cf0-4f6e-84f8-0a78053f72ec service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:20.693416 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:20.694296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.694699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.695188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.696546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:20.698953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:20.752131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1376/2771] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:20 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:20.758165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:20.775593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c84ba1-0582-42dd-9932-363d0a3794ed None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:20.909203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:20.910100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.910487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.910866 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.911709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:20.913945 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:20.976446 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1396/2772] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:20 2022] POST /identity/v3/users => generated 295 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:39:20.982041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:21.004282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edca46-3ab9-4cfc-b149-b3bc2482ca04 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:21.013036 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:21.015555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:21.016436 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:21.017195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:21.018814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:21.022248 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:21.048193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1377/2773] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:21.056422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:21.078810 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e5c66-ed9a-47d2-b56b-82553239df3e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:21.087141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:21.089046 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:21.089749 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:21.090388 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] PATH_INFO: `/v3/projects/53b2dc9e01c9411f8253637fb8439dbd/users/53d69d0d5b6847189d3f84975aad6742/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:21.126042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b2dc9e01c9411f8253637fb8439dbd, user_id=53d69d0d5b6847189d3f84975aad6742, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:21.131836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:21.164119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1397/2774] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:21 2022] PUT /identity/v3/projects/53b2dc9e01c9411f8253637fb8439dbd/users/53d69d0d5b6847189d3f84975aad6742/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:39:21.169970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:21.196612 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08d083-348e-4e26-a0b6-c17efc738705 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:21.316647 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:21.317843 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:21.318317 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:21.318765 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:21.320155 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:21.323247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:21.348267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1378/2775] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:21.350918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:21.366857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966ab2c-785e-4d18-92ab-02d934f5b24e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:21.372470 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:21.373433 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:21.373719 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:21.374007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] PATH_INFO: `/v3/projects/53b2dc9e01c9411f8253637fb8439dbd/users/53d69d0d5b6847189d3f84975aad6742/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:21.376898 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b2dc9e01c9411f8253637fb8439dbd, user_id=53d69d0d5b6847189d3f84975aad6742, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:21.379940 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:21.395122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1398/2776] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:21 2022] PUT /identity/v3/projects/53b2dc9e01c9411f8253637fb8439dbd/users/53d69d0d5b6847189d3f84975aad6742/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:22.126005 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:22.130336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:22.251801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b541-c373-489e-b77d-f4235106f99f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:22.349653 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:22.351709 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:22.352527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:22.353280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:22.356834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:22.378503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:22.381181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f958b9-62d0-40a6-8c2a-55b86d933955 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:22.388470 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:22.414413 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1379/2777] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:29.737822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:29.745222 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:30.023097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ecbf-da2a-4568-af49-2fa040b6777d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:30.038926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:30.040101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:30.040666 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:30.041631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:30.044817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:30.067801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:30.070760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:30.089740 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:30.107917 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1399/2778] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:30.413744 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:30.414257 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:30.414775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:30.457194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] MFA Rules not processed for user `53d69d0d5b6847189d3f84975aad6742`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:30.458480 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:30.628954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1380/2779] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:30.668157 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:30.669621 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:30.722052 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d112-7ed9-4d2d-870d-fdca76778140 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:30.736524 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:30.737925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:30.740888 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:30.741750 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:30.890978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:30.894141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:30.911732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1400/2780] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:39:36.197851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:36.215452 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdfb18-b88c-47a1-b43e-1fbbd4c7db4b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:36.221055 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:36.221763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:36.222051 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:36.222332 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:36.224423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:36.313512 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:36.315034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:36.327811 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1381/2781] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:40.901868 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=28 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:40.957367 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:40.959819 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:40.975526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed1b16-0862-428d-a2c3-32958859e0d3 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:40.982227 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:40.983800 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:40.984360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:40.984844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] PATH_INFO: `/v3/users/ebdf8e3cd1b44dcba37cdcfb4af7db2b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:40.987179 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdf8e3cd1b44dcba37cdcfb4af7db2b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:40.990937 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:41.063817 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdf8e3cd1b44dcba37cdcfb4af7db2b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.083700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdf8e3cd1b44dcba37cdcfb4af7db2b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.092867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdf8e3cd1b44dcba37cdcfb4af7db2b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.093768 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdf8e3cd1b44dcba37cdcfb4af7db2b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.094614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-275c8684-e0f5-4e6e-a14e-a98c05c6ed1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdf8e3cd1b44dcba37cdcfb4af7db2b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.113243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1401/2782] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:40 2022] DELETE /identity/v3/users/ebdf8e3cd1b44dcba37cdcfb4af7db2b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:41.368716 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:41.383301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1b6a7-c305-4313-9637-936946d6b39f service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:41.491142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:41.493264 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.493763 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.494222 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] PATH_INFO: `/v3/projects/7a8e1d26bdfc400ebd1e070b09db615e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.496606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8e1d26bdfc400ebd1e070b09db615e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:41.501285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:41.546234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:41.583460 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8e1d26bdfc400ebd1e070b09db615e', 'request_id': 'req-ae70e87b-7d7d-4814-8f94-48abb7038fa5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.591989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8e1d26bdfc400ebd1e070b09db615e', 'request_id': 'req-ae70e87b-7d7d-4814-8f94-48abb7038fa5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.600430 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8e1d26bdfc400ebd1e070b09db615e', 'request_id': 'req-ae70e87b-7d7d-4814-8f94-48abb7038fa5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.616226 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] The token cache is being invalidate because project 7a8e1d26bdfc400ebd1e070b09db615e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:41.616526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.617200 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ae70e87b-7d7d-4814-8f94-48abb7038fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:41.618840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1382/2783] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:41 2022] DELETE /identity/v3/projects/7a8e1d26bdfc400ebd1e070b09db615e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:41.624593 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.625094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.625487 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.650805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:41.652371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:41.730646 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1402/2784] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:41.744602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.744977 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.745323 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.767513 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:41.768819 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:41.777982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1383/2785] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:41.783393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:41.794417 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bb5b6-5ad8-4cc0-b889-016409fbccef None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:41.801496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:41.802199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.802480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.802751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.803499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:41.803832 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:41.805515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:41.822158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1403/2786] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:41.828787 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:41.839927 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f875e-0042-401d-b61d-bd4564c18123 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:41.847528 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:41.848529 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.848928 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.849338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.850097 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:41.852246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:41.893127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1384/2787] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:41 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:41.899392 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:41.915105 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00bda-310e-430f-961a-4d34a5ad6f2e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:41.994843 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:41.995505 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.995790 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.996087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.996816 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:41.999368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.007050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:42.026939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1404/2788] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:41 2022] POST /identity/v3/users => generated 282 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:42.033850 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.046804 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8efdc-e926-42c3-93bf-751afd40cab4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.052986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.053786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.054081 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.054355 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.055142 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.056833 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.070023 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1385/2789] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:42.074586 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.087549 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae1a21-42f8-4958-b1ba-2772f6431a46 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.094492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db362101-4b36-405a-9d3e-cea00cb07881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.095760 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362101-4b36-405a-9d3e-cea00cb07881 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.096188 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362101-4b36-405a-9d3e-cea00cb07881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.096633 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362101-4b36-405a-9d3e-cea00cb07881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.097699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db362101-4b36-405a-9d3e-cea00cb07881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.100253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db362101-4b36-405a-9d3e-cea00cb07881 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.120187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1405/2790] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:39:42.125347 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.136605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15177758-d674-4c88-8287-dd717cede951 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.143075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.144035 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.144361 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.144646 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] PATH_INFO: `/v3/projects/77d43b53f6f04b7e9260772c3845973f/users/baa565aca9154253b237402335ddbddd/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.165762 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d43b53f6f04b7e9260772c3845973f, user_id=baa565aca9154253b237402335ddbddd, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.168940 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.188892 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1386/2791] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:42 2022] PUT /identity/v3/projects/77d43b53f6f04b7e9260772c3845973f/users/baa565aca9154253b237402335ddbddd/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:42.206033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.206315 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.206697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.242103 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] MFA Rules not processed for user `baa565aca9154253b237402335ddbddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:42.243372 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:42.328694 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1406/2792] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:42.335683 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.430776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ecf5d-8dff-4b8f-be6f-9fb8defb450b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.437661 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.438620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.439004 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.440375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.440375 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.442234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.477107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1387/2793] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:42 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:42.481332 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.495670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f9f06-14d9-4b89-9e2a-5039e3b46c0b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.588494 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.589119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.589410 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.589693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.590447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.592688 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.621031 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1407/2794] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:42 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:42.627630 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.643490 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc0a0-7a18-4d5b-808a-b21218767691 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.649807 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.651108 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.651606 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.652195 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.653325 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.656604 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.671234 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1388/2795] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:42.675967 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.687986 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc9458-2b0f-490e-bc18-9ac9fb68e0c6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.696087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.697928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.698503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.698951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] PATH_INFO: `/v3/projects/8355015acfbc472eaa23c80c5ae3f594/users/1b80f36b48c24eaba43c93f01c9ff44d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.720466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8355015acfbc472eaa23c80c5ae3f594, user_id=1b80f36b48c24eaba43c93f01c9ff44d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.723187 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.743763 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1408/2796] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:42 2022] PUT /identity/v3/projects/8355015acfbc472eaa23c80c5ae3f594/users/1b80f36b48c24eaba43c93f01c9ff44d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:42.748381 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.761439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22629e17-4777-4674-84f0-ab530998d22e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.783712 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:42.848962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812648c1-f872-49d0-aef3-99b4e88f85a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.849674 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812648c1-f872-49d0-aef3-99b4e88f85a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.849963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812648c1-f872-49d0-aef3-99b4e88f85a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.850246 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812648c1-f872-49d0-aef3-99b4e88f85a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.851006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812648c1-f872-49d0-aef3-99b4e88f85a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.852708 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812648c1-f872-49d0-aef3-99b4e88f85a5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.865864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1389/2797] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:42.870219 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:42.882526 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f73f52-5c56-4a34-9fbb-20c06037715b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:42.887691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:42.888652 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.888931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.889836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] PATH_INFO: `/v3/projects/8355015acfbc472eaa23c80c5ae3f594/users/1b80f36b48c24eaba43c93f01c9ff44d/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.892101 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8355015acfbc472eaa23c80c5ae3f594, user_id=1b80f36b48c24eaba43c93f01c9ff44d, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:42.894794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:42.910191 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1409/2798] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:42 2022] PUT /identity/v3/projects/8355015acfbc472eaa23c80c5ae3f594/users/1b80f36b48c24eaba43c93f01c9ff44d/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:42.923450 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:42.923733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:42.924287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:42.957154 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] MFA Rules not processed for user `1b80f36b48c24eaba43c93f01c9ff44d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:42.958568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:43.048678 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1390/2799] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:43.062130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:43.064992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:43.092983 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:43.141334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b2b82-35fc-480f-9c38-cf6208a101ec admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:43.145399 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6225f-951f-4112-ad6c-dfdf1646b648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:43.146141 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6225f-951f-4112-ad6c-dfdf1646b648 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.146499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6225f-951f-4112-ad6c-dfdf1646b648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.146890 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6225f-951f-4112-ad6c-dfdf1646b648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.158572 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6225f-951f-4112-ad6c-dfdf1646b648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:43.160448 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6225f-951f-4112-ad6c-dfdf1646b648 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:43.173350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1410/2800] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:43.761640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:43.854218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568efd44-bd3e-418b-ac3d-1da7012fb00d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:43.860527 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:43.861443 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.861730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.862009 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] PATH_INFO: `/v3/users/baa565aca9154253b237402335ddbddd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.863407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa565aca9154253b237402335ddbddd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:43.865803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:43.942021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:43.949803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa565aca9154253b237402335ddbddd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:43.959566 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa565aca9154253b237402335ddbddd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:43.970217 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa565aca9154253b237402335ddbddd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:43.993197 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa565aca9154253b237402335ddbddd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:43.994247 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa565aca9154253b237402335ddbddd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:44.005886 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1391/2801] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:43 2022] DELETE /identity/v3/users/baa565aca9154253b237402335ddbddd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:39:44.012036 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:44.012597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:44.013113 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:44.044404 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:44.047097 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:44.164552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1411/2802] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:44.178865 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:44.182504 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:44.296380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbe05f-086b-4e7b-97ee-3d38dd371a3f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:44.304250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:44.305582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:44.306286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:44.306775 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:44.327992 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:44.330995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:44.352565 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1392/2803] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:44.733322 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:44.735918 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:44.762428 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf58d3e-b9d6-4035-bb40-30ff3793fa90 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:44.772226 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:44.773777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:44.774305 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:44.774818 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] PATH_INFO: `/v3/projects/77d43b53f6f04b7e9260772c3845973f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:44.777158 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d43b53f6f04b7e9260772c3845973f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:44.781565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:44.869831 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d43b53f6f04b7e9260772c3845973f', 'request_id': 'req-b3fbf216-9734-4d9b-b527-c89c6bae06bf'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:44.878121 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d43b53f6f04b7e9260772c3845973f', 'request_id': 'req-b3fbf216-9734-4d9b-b527-c89c6bae06bf'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:44.886687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d43b53f6f04b7e9260772c3845973f', 'request_id': 'req-b3fbf216-9734-4d9b-b527-c89c6bae06bf'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:44.906938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] The token cache is being invalidate because project 77d43b53f6f04b7e9260772c3845973f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:44.907382 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:44.908307 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:44.910314 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1412/2804] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:44 2022] DELETE /identity/v3/projects/77d43b53f6f04b7e9260772c3845973f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:44.914779 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:44.926140 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c488e8f-be59-48b1-b278-0e7e2f2452e3 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:45.033468 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:45.034947 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:45.035631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:45.036169 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] PATH_INFO: `/v3/users/1b80f36b48c24eaba43c93f01c9ff44d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:45.038377 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b80f36b48c24eaba43c93f01c9ff44d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:45.042266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:45.105717 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b80f36b48c24eaba43c93f01c9ff44d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.114798 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b80f36b48c24eaba43c93f01c9ff44d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.125560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b80f36b48c24eaba43c93f01c9ff44d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.127073 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:45.143644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b80f36b48c24eaba43c93f01c9ff44d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.144439 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2fc152ff-52ee-429a-b987-f1b77a4c0a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b80f36b48c24eaba43c93f01c9ff44d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.151622 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1393/2805] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:44 2022] DELETE /identity/v3/users/1b80f36b48c24eaba43c93f01c9ff44d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:45.495017 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:45.497792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:45.520531 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbf216-9734-4d9b-b527-c89c6bae06bf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:45.588931 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:45.607171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:45.609435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:45.609727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:45.610008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] PATH_INFO: `/v3/projects/8355015acfbc472eaa23c80c5ae3f594` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:45.611443 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8355015acfbc472eaa23c80c5ae3f594)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:45.613781 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:45.675069 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8355015acfbc472eaa23c80c5ae3f594', 'request_id': 'req-51027a50-9e05-4f1f-899d-d99a21247cb3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.682727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8355015acfbc472eaa23c80c5ae3f594', 'request_id': 'req-51027a50-9e05-4f1f-899d-d99a21247cb3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.691566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8355015acfbc472eaa23c80c5ae3f594', 'request_id': 'req-51027a50-9e05-4f1f-899d-d99a21247cb3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.713274 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] The token cache is being invalidate because project 8355015acfbc472eaa23c80c5ae3f594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:45.713670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.714523 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51027a50-9e05-4f1f-899d-d99a21247cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:45.716677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1413/2806] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:45 2022] DELETE /identity/v3/projects/8355015acfbc472eaa23c80c5ae3f594 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:45.722854 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:45.723146 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:45.723438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:45.752505 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:45.754697 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:45.873407 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1394/2807] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:45.884089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:45.884622 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:45.885122 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:45.917030 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:45.919225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:45.933145 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1414/2808] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:45.938835 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:45.950517 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0362-b683-4ab3-b37d-8937248200ac None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:45.956104 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:45.956805 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:45.957159 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:45.957580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:45.958580 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:45.958997 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:45.961481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:45.976399 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1395/2809] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:45.981791 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:45.992529 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc6e27-bb34-4fc0-aaea-12831f574cc4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:45.997767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:45.999333 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:45.999828 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.000357 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.001155 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.003477 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.036825 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1415/2810] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:45 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:46.041087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.054839 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d32616-2a18-4f1c-b7df-38e9af512ac9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.132030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.132675 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.132980 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.133235 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.133953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.136006 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.160953 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1396/2811] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:46 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:39:46.165916 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.179629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e51220-e118-4942-8b15-9404aabf644c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.185277 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.187108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.187540 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.187941 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.189013 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.190772 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.204903 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1416/2812] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:46.210041 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.222718 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1247a-9623-41b7-9f64-e7dae2de1df0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.227733 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb28f08-4826-418c-93fb-e63483c730e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.228551 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28f08-4826-418c-93fb-e63483c730e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.228842 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28f08-4826-418c-93fb-e63483c730e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.229127 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28f08-4826-418c-93fb-e63483c730e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.229889 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb28f08-4826-418c-93fb-e63483c730e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.231546 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb28f08-4826-418c-93fb-e63483c730e4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.246133 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1397/2813] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:39:46.251218 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.263907 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d242286-e507-4fcb-a458-cbcd3064a76c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.270213 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.271290 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.271667 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.272067 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] PATH_INFO: `/v3/projects/a56438f88bfe44f5859c056b5abf665b/users/103a82736dd3443ea9b6d373a0770dd8/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.291260 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56438f88bfe44f5859c056b5abf665b, user_id=103a82736dd3443ea9b6d373a0770dd8, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.294350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.314499 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1417/2814] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:46 2022] PUT /identity/v3/projects/a56438f88bfe44f5859c056b5abf665b/users/103a82736dd3443ea9b6d373a0770dd8/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:46.326003 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.326351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.326817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.361084 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] MFA Rules not processed for user `103a82736dd3443ea9b6d373a0770dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:46.362349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:46.411840 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.466620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1398/2815] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:46.472863 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.592583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba7a2-4bd7-49bb-97d5-9112ac252ba6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.601016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.602156 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.602624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.603375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.604877 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.609453 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.643350 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.657054 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1418/2816] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:46 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:46.661144 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.674298 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89542dc2-568f-4052-9fc2-62c81a331445 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.754129 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.754829 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.755116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.755447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.756345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.758339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.787603 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1399/2817] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:46 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:46.793437 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.806396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344c63-c254-4f87-9e0e-4140d129f9d0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.811881 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.812648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.812926 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.813210 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.813973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.815616 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.831503 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1419/2818] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:46.836295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.848806 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34273a0-e109-4b04-b225-a0d830a49b30 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.853874 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.854814 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.855099 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.855554 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] PATH_INFO: `/v3/projects/fd789a903ca44a069b2d49399a4bec83/users/846ce966898a4bd38c82185221cb7d33/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.874639 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd789a903ca44a069b2d49399a4bec83, user_id=846ce966898a4bd38c82185221cb7d33, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.877542 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.894126 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1400/2819] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:46 2022] PUT /identity/v3/projects/fd789a903ca44a069b2d49399a4bec83/users/846ce966898a4bd38c82185221cb7d33/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:46.899008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.911359 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fa86a-75f5-4c6e-b48d-25578d6b78cb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.983809 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42683a-961e-450a-b6f0-42df1dc89385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.984550 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42683a-961e-450a-b6f0-42df1dc89385 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.984841 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42683a-961e-450a-b6f0-42df1dc89385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.985120 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42683a-961e-450a-b6f0-42df1dc89385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.985885 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42683a-961e-450a-b6f0-42df1dc89385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.987552 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42683a-961e-450a-b6f0-42df1dc89385 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.999939 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1420/2820] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:47.004771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:47.016753 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7b110-05fc-4a2f-9141-3107c72d2a7e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:47.022241 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:47.023370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.023700 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.024096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] PATH_INFO: `/v3/projects/fd789a903ca44a069b2d49399a4bec83/users/846ce966898a4bd38c82185221cb7d33/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.026560 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd789a903ca44a069b2d49399a4bec83, user_id=846ce966898a4bd38c82185221cb7d33, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:47.029477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:47.043284 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1401/2821] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:47 2022] PUT /identity/v3/projects/fd789a903ca44a069b2d49399a4bec83/users/846ce966898a4bd38c82185221cb7d33/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:39:47.056237 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.056853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.057247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.092568 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] MFA Rules not processed for user `846ce966898a4bd38c82185221cb7d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:47.094050 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:47.186695 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1421/2822] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:47.201995 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:47.204337 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:47.296318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7f48c-3fd3-4ffb-9f3c-d2abf8184a42 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:47.304174 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:47.305349 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.305875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.306391 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.326164 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:47.330189 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:47.346687 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1402/2823] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:39:47.468919 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:47.479476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e724f02-6431-4cbd-a316-a30f25e39443 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:47.484347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a4f1c-ade9-4c4d-bc46-f2a726aecece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:47.485336 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a4f1c-ade9-4c4d-bc46-f2a726aecece service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.485963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a4f1c-ade9-4c4d-bc46-f2a726aecece service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.486375 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a4f1c-ade9-4c4d-bc46-f2a726aecece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.565685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a4f1c-ade9-4c4d-bc46-f2a726aecece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:47.567165 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a4f1c-ade9-4c4d-bc46-f2a726aecece service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:47.579267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1422/2824] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:47.937514 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:47.974685 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e83d60-5b91-405a-ad47-9face207c72f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.975510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e83d60-5b91-405a-ad47-9face207c72f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.975510 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e83d60-5b91-405a-ad47-9face207c72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.982676 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:47.992509 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:48.001610 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d1e83d60-5b91-405a-ad47-9face207c72f None None] MFA Rules not processed for user `e5cbf8d443de4a9fb627ef316fbd033f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:48.003131 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d1e83d60-5b91-405a-ad47-9face207c72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:48.022206 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab459b-0c36-4647-b436-71fdb7dae56f service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:48.027783 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:48.028726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:48.029032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:48.029292 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] PATH_INFO: `/v3/users/103a82736dd3443ea9b6d373a0770dd8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:48.031489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] RBAC: Authorizing `identity:delete_user(user_id=103a82736dd3443ea9b6d373a0770dd8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:48.033540 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:48.081506 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1423/2825] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:48.098585 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103a82736dd3443ea9b6d373a0770dd8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:48.108624 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103a82736dd3443ea9b6d373a0770dd8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:48.119703 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103a82736dd3443ea9b6d373a0770dd8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:48.144942 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103a82736dd3443ea9b6d373a0770dd8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:48.146242 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103a82736dd3443ea9b6d373a0770dd8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:48.155534 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1403/2826] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:47 2022] DELETE /identity/v3/users/103a82736dd3443ea9b6d373a0770dd8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:48.161182 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:48.161501 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:48.161737 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:48.184364 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:48.185634 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:48.264563 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1424/2827] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:48.272124 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:48.274981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:48.348285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125e13-3b5a-4dae-b31b-029f025b002a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:48.353367 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:48.354083 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:48.354372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:48.354664 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:48.365973 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:48.368031 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:48.379632 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1404/2828] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:48.480511 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:48.482393 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:48.499193 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd3d9-8fdd-4461-a4c3-1e37c4767e51 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:48.504543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:48.505626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:48.506108 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:48.506565 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:48.509600 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:48.525480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:48.527171 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:48.537725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1425/2829] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:48.890875 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:48.892911 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:48.915459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0acea2-91ce-4d6c-a53c-02b452b049e8 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:48.925353 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:48.926431 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:48.926837 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:48.927194 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] PATH_INFO: `/v3/projects/a56438f88bfe44f5859c056b5abf665b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:48.928950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56438f88bfe44f5859c056b5abf665b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:48.931817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:49.024338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:49.026296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56438f88bfe44f5859c056b5abf665b', 'request_id': 'req-1db3a6d1-b596-4e62-b2b7-eef98e84957e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.037072 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56438f88bfe44f5859c056b5abf665b', 'request_id': 'req-1db3a6d1-b596-4e62-b2b7-eef98e84957e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.051684 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56438f88bfe44f5859c056b5abf665b', 'request_id': 'req-1db3a6d1-b596-4e62-b2b7-eef98e84957e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.074111 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] The token cache is being invalidate because project a56438f88bfe44f5859c056b5abf665b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:49.074419 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.075110 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.077494 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1405/2830] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:48 2022] DELETE /identity/v3/projects/a56438f88bfe44f5859c056b5abf665b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:49.082312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:49.101039 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027928b-a731-4edc-9883-03ce5aed97d3 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:49.189475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:49.190380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:49.190668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:49.190974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] PATH_INFO: `/v3/users/846ce966898a4bd38c82185221cb7d33` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:49.192396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=846ce966898a4bd38c82185221cb7d33)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:49.194351 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:49.262639 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:49.274224 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '846ce966898a4bd38c82185221cb7d33'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.280266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:49.283915 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:49.303186 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '846ce966898a4bd38c82185221cb7d33'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.319797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '846ce966898a4bd38c82185221cb7d33'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.321173 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '846ce966898a4bd38c82185221cb7d33'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.322252 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '846ce966898a4bd38c82185221cb7d33'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.357767 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1426/2831] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:39:49 2022] DELETE /identity/v3/users/846ce966898a4bd38c82185221cb7d33 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:49.404456 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3a6d1-b596-4e62-b2b7-eef98e84957e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:49.487761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:49.488858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:49.489188 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:49.489623 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:49.492134 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:49.510547 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:49.512238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d14270-da2c-45d0-bf18-2be7550bc2a0 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:49.525588 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1406/2832] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:49.765849 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:49.768878 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:49.888770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7f77f-33cd-4c60-96c3-a1b25ccf27f0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:49.896133 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:49.897724 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:49.898225 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:49.898705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] PATH_INFO: `/v3/projects/fd789a903ca44a069b2d49399a4bec83` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:49.901284 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd789a903ca44a069b2d49399a4bec83)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:49.905882 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:49.982134 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd789a903ca44a069b2d49399a4bec83', 'request_id': 'req-c27089a7-6571-4dfa-9038-9eae8893747f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:49.994024 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd789a903ca44a069b2d49399a4bec83', 'request_id': 'req-c27089a7-6571-4dfa-9038-9eae8893747f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:50.003126 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd789a903ca44a069b2d49399a4bec83', 'request_id': 'req-c27089a7-6571-4dfa-9038-9eae8893747f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:50.024301 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] The token cache is being invalidate because project fd789a903ca44a069b2d49399a4bec83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:39:50.024794 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:50.025987 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c27089a7-6571-4dfa-9038-9eae8893747f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:39:50.028775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1427/2833] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:39:49 2022] DELETE /identity/v3/projects/fd789a903ca44a069b2d49399a4bec83 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:50.036459 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.037276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.037276 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.059608 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:50.061278 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:50.143877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1407/2834] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:50.158180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.159099 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.159299 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.181994 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:50.183287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:50.193231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1428/2835] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:50.197976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.211338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a322b-c400-4c53-a1af-e883e40831bd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.216519 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.217265 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.217582 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.217818 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.218584 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:50.218903 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.220740 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.223847 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:50.246848 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1408/2836] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:39:50.249643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.264965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6c424-8b67-4353-8f79-59d724731dd9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.273087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.273851 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.274177 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.274497 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.275342 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.277517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.319974 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1429/2837] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:50 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:50.324525 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.336438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b7d92-50c5-4832-ac7a-0323bfd41663 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.405690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.406422 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.406749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.406983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.407705 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.409738 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.438916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1409/2838] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:50 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:39:50.440461 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.455137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc034c-55d3-4b49-98ef-7ccbe537f870 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.462276 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.463234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.463647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.463993 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.464971 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.466853 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.480247 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1430/2839] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:50.485319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.496902 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e18d32-8bfd-4f8b-800f-ed1301755f1f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.505395 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8ac8b-fcab-4b9a-941a-a528249ef3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.505851 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ac8b-fcab-4b9a-941a-a528249ef3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.506240 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ac8b-fcab-4b9a-941a-a528249ef3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.506617 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ac8b-fcab-4b9a-941a-a528249ef3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.507908 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8ac8b-fcab-4b9a-941a-a528249ef3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.509972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8ac8b-fcab-4b9a-941a-a528249ef3ad admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.526690 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1410/2840] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:50.533668 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.545253 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b496-c7a1-4dcc-a28a-041be74e123a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.551538 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.552561 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.552850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.553130 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] PATH_INFO: `/v3/projects/7f5223c7d6e246d7ac1396493caa0de4/users/c9a09cf4dd474c1285455ffa141c5f7e/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.558636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:50.579547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5223c7d6e246d7ac1396493caa0de4, user_id=c9a09cf4dd474c1285455ffa141c5f7e, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.582812 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.601135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1431/2841] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:50 2022] PUT /identity/v3/projects/7f5223c7d6e246d7ac1396493caa0de4/users/c9a09cf4dd474c1285455ffa141c5f7e/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:39:50.607686 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a36283-cbc0-41e1-893c-93c525628e98 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.608620 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a36283-cbc0-41e1-893c-93c525628e98 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.608912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a36283-cbc0-41e1-893c-93c525628e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.633374 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c9a36283-cbc0-41e1-893c-93c525628e98 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:50.635252 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c9a36283-cbc0-41e1-893c-93c525628e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:50.724151 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1411/2842] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:50.739911 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:50.742245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:50.840041 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983515-5ea0-475f-bd57-1cc66e498841 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:50.846541 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:50.847273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:50.847651 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:50.848128 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:50.863675 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:50.865580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:50.879753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1432/2843] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:51.997495 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:51.998986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:51.998986 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.036256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] MFA Rules not processed for user `c9a09cf4dd474c1285455ffa141c5f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:52.038223 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:52.141328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1412/2844] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:39:52.142530 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:52.155092 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2587c8-2c83-42d5-9fe8-61cca377f493 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:52.160897 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:52.161770 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:52.162266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:52.162732 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.163660 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:52.165700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:52.220709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1433/2845] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:39:52 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:39:52.226280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:52.238119 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:52.246424 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9610-139f-4c3a-986f-525bc2a65c13 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:52.383048 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:52.384294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:52.384780 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:52.385267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.386611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:52.390537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:52.425963 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1413/2846] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:39:52 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:52.431704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:52.450883 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bee7d-fe39-4e26-8553-d5374eb3a46d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:52.459921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:52.461135 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:52.461636 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:52.462146 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.463587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:52.468180 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:52.495756 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1434/2847] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:39:52.497186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:52.511968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067cec5-4145-4fc7-8cb7-bac264d9ffb2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:52.519182 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:52.522933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:52.523294 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:52.523502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] PATH_INFO: `/v3/projects/7176e575c91243b88d9749409d87a838/users/b1f4024fe66a4c5e8be43bc3a2817e6b/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.543372 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7176e575c91243b88d9749409d87a838, user_id=b1f4024fe66a4c5e8be43bc3a2817e6b, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:52.546931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:52.570360 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1414/2848] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:52 2022] PUT /identity/v3/projects/7176e575c91243b88d9749409d87a838/users/b1f4024fe66a4c5e8be43bc3a2817e6b/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:52.577401 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:52.595119 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d3f41-a20c-446a-9eec-d3ccc58ef9e9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:52.690360 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11199262-3777-49ff-9db3-0fdd7a94ce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:52.690955 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11199262-3777-49ff-9db3-0fdd7a94ce23 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:52.691242 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11199262-3777-49ff-9db3-0fdd7a94ce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:52.691520 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11199262-3777-49ff-9db3-0fdd7a94ce23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.692345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11199262-3777-49ff-9db3-0fdd7a94ce23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:52.694071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11199262-3777-49ff-9db3-0fdd7a94ce23 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:52.712484 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1435/2849] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:39:52.716976 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:52.733458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27d66e-df24-4cd8-adde-b00019f71df4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:52.740021 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:52.741216 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:52.741552 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:52.741870 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] PATH_INFO: `/v3/projects/7176e575c91243b88d9749409d87a838/users/b1f4024fe66a4c5e8be43bc3a2817e6b/roles/bd65b447e4544528b91e8a65a4f8ee54` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:52.755877 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7176e575c91243b88d9749409d87a838, user_id=b1f4024fe66a4c5e8be43bc3a2817e6b, role_id=bd65b447e4544528b91e8a65a4f8ee54)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:52.761269 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:52.778402 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1415/2850] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:39:52 2022] PUT /identity/v3/projects/7176e575c91243b88d9749409d87a838/users/b1f4024fe66a4c5e8be43bc3a2817e6b/roles/bd65b447e4544528b91e8a65a4f8ee54 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:53.983559 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:53.984282 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:53.984391 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:54.022117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] MFA Rules not processed for user `b1f4024fe66a4c5e8be43bc3a2817e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:54.023283 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:54.087904 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:54.134714 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1436/2851] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:39:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:39:54.147440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:54.149299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:54.265106 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7de8a-7b3d-4573-812b-2dc7c000a57b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:54.269282 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:54.269970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:54.270250 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:54.270696 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:54.356296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:54.358432 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:54.376302 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1416/2852] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:39:55.935368 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:55.938266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:55.959579 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a8c4-e8bb-4ceb-9791-339ee2eb11a2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:55.967699 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:55.969543 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:55.970294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:55.971056 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:55.993115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:55.996730 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:56.019644 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1437/2853] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:39:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:40:04.366706 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:05.345017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:05.347438 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:05.471236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5c0f-84c9-465a-8c01-f5cf9803d81e service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:05.478239 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:05.478972 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:05.479295 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:05.479613 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:05.481695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:05.497844 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:05.501017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:05.515392 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1417/2854] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:40:06.010104 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=17 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:08.844385 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:08.913371 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a23380-6563-4eb7-a7e3-da975342bfc0 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:08.917583 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:08.918262 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:08.918547 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:08.918824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:08.921736 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:09.051648 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:09.053331 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:09.069369 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1438/2855] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:40:10.731318 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:10.819116 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31636b75-894f-4296-8d77-75dcc73b4065 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:10.824473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:10.825408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:10.825688 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:10.826019 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] PATH_INFO: `/v3/users/c9a09cf4dd474c1285455ffa141c5f7e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:10.827526 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a09cf4dd474c1285455ffa141c5f7e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:10.829576 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:10.918968 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a09cf4dd474c1285455ffa141c5f7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:10.932394 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a09cf4dd474c1285455ffa141c5f7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:10.945749 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a09cf4dd474c1285455ffa141c5f7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:10.963193 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a09cf4dd474c1285455ffa141c5f7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:10.964087 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a09cf4dd474c1285455ffa141c5f7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:10.971768 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1418/2856] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:40:10 2022] DELETE /identity/v3/users/c9a09cf4dd474c1285455ffa141c5f7e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:40:11.314604 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:11.331874 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c8354-e84e-4f7a-9600-da0555b04b21 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:11.444908 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:11.446142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:11.446466 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:11.446822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] PATH_INFO: `/v3/projects/7f5223c7d6e246d7ac1396493caa0de4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:11.448258 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5223c7d6e246d7ac1396493caa0de4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:11.450459 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:11.537203 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5223c7d6e246d7ac1396493caa0de4', 'request_id': 'req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.539857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:11.546775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5223c7d6e246d7ac1396493caa0de4', 'request_id': 'req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.555380 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5223c7d6e246d7ac1396493caa0de4', 'request_id': 'req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.571089 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] The token cache is being invalidate because project 7f5223c7d6e246d7ac1396493caa0de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:40:11.571445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.572496 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.574946 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1439/2857] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:40:11 2022] DELETE /identity/v3/projects/7f5223c7d6e246d7ac1396493caa0de4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:40:11.581067 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:11.583611 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:11.600481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:11.611677 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d4f4b-8606-42df-bc7c-ba1cd4805274 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:11.724832 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:11.725567 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:11.725985 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:11.726415 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] PATH_INFO: `/v3/users/b1f4024fe66a4c5e8be43bc3a2817e6b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:11.728634 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f4024fe66a4c5e8be43bc3a2817e6b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:11.731524 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:11.757785 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:11.761468 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:11.832766 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f4024fe66a4c5e8be43bc3a2817e6b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.843319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f4024fe66a4c5e8be43bc3a2817e6b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.855359 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f4024fe66a4c5e8be43bc3a2817e6b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.878089 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f4024fe66a4c5e8be43bc3a2817e6b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.879378 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f4024fe66a4c5e8be43bc3a2817e6b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:11.888141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1419/2858] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:40:11 2022] DELETE /identity/v3/users/b1f4024fe66a4c5e8be43bc3a2817e6b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:40:11.913700 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ad50a-6c07-4557-99a5-c76d5e6dc6d1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:12.042970 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:12.044325 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:12.044934 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:12.045624 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:12.051705 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:12.078928 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:12.082480 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3939-378d-4537-bec5-10bfc2cbdc2b service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:12.109871 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1440/2859] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:40:12.173849 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:12.176287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:12.318853 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156e2b-31fa-435b-873d-3d723024c296 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:12.333386 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:12.334410 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:12.334695 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:12.334969 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] PATH_INFO: `/v3/projects/7176e575c91243b88d9749409d87a838` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:12.336485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7176e575c91243b88d9749409d87a838)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:12.339724 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:12.396408 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:12.431148 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7176e575c91243b88d9749409d87a838', 'request_id': 'req-f54937cb-1719-4658-abdd-bccc9018509a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:12.441283 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7176e575c91243b88d9749409d87a838', 'request_id': 'req-f54937cb-1719-4658-abdd-bccc9018509a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:12.452935 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7176e575c91243b88d9749409d87a838', 'request_id': 'req-f54937cb-1719-4658-abdd-bccc9018509a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:12.477591 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] The token cache is being invalidate because project 7176e575c91243b88d9749409d87a838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:40:12.478370 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:12.480113 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f54937cb-1719-4658-abdd-bccc9018509a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:12.482272 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1420/2860] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:40:12 2022] DELETE /identity/v3/projects/7176e575c91243b88d9749409d87a838 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:40:12.489395 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:12.490243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:12.490447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:12.526625 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:12.528846 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:12.646184 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1441/2861] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:40:12.657249 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:12.657771 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:12.658256 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:12.686502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:12.687933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:12.698137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1421/2862] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:40:12.704670 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:12.716199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:12.728263 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046e89-9ad3-4024-89e1-9fbd13b521e5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:12.752999 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:12.755716 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:12.756711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:12.758472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:12.761850 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:40:12.762687 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:12.766822 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:12.795445 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1442/2863] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:40:12.802912 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:12.821836 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1da45-0d2d-470f-8855-1e60c5e05b38 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:12.830508 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:12.831786 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:12.832444 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:12.832950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:12.834267 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:12.838131 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:12.891500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1422/2864] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:40:12 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:40:12.896142 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:12.915834 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963f544-fee5-40e1-8a1d-d346a26ec33c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.034183 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.034978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.035259 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.035536 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.036318 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.038312 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:13.061978 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1443/2865] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:40:12 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:40:13.066982 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:13.082214 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101533c2-e87b-4dc9-8796-a4578f9b75a7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.089380 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.090179 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.090555 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.091027 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.094932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.098483 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:13.118388 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1423/2866] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:40:13.124665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:13.140232 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd625-f54c-40d6-89c3-dd6a6caf67b8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.146289 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9b199-8c71-4f27-8c81-2bd637847c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.146992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b199-8c71-4f27-8c81-2bd637847c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.147279 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b199-8c71-4f27-8c81-2bd637847c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.147556 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b199-8c71-4f27-8c81-2bd637847c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.148349 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b199-8c71-4f27-8c81-2bd637847c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.150023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b199-8c71-4f27-8c81-2bd637847c9e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:13.162795 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1444/2867] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:40:13.167301 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:13.180471 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872030e7-9b1f-40af-a13d-ec05669bc3b5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.187406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.188379 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.188663 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.188933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] PATH_INFO: `/v3/projects/58c4e094a3694449a93f7d39aee0187e/users/2ad404cc45a74432b83dcc1836771b66/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.211726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c4e094a3694449a93f7d39aee0187e, user_id=2ad404cc45a74432b83dcc1836771b66, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.214614 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:13.238420 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1424/2868] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:40:13 2022] PUT /identity/v3/projects/58c4e094a3694449a93f7d39aee0187e/users/2ad404cc45a74432b83dcc1836771b66/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:40:13.250952 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.251472 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.251855 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.289344 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] MFA Rules not processed for user `2ad404cc45a74432b83dcc1836771b66`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:13.290597 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:13.397533 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1445/2869] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:40:13.410713 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:13.412575 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:13.487954 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be496b0-e238-40fa-81d0-2fd1782593e7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.492163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de5ccd-0fc0-4887-8208-69ec7584be9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.492864 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de5ccd-0fc0-4887-8208-69ec7584be9c service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.493149 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de5ccd-0fc0-4887-8208-69ec7584be9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.493423 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de5ccd-0fc0-4887-8208-69ec7584be9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.503702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de5ccd-0fc0-4887-8208-69ec7584be9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.505681 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de5ccd-0fc0-4887-8208-69ec7584be9c service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:13.517666 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1425/2870] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:40:13.538827 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:13.643745 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec02d4-1001-4196-a386-4e2c74dd3490 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.657842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.659778 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.661408 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.661927 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] PATH_INFO: `/v3/users/2ad404cc45a74432b83dcc1836771b66` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.665614 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad404cc45a74432b83dcc1836771b66)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.671176 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:13.674330 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:13.732042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad404cc45a74432b83dcc1836771b66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:13.746094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad404cc45a74432b83dcc1836771b66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:13.754551 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad404cc45a74432b83dcc1836771b66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:13.755321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad404cc45a74432b83dcc1836771b66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:13.755938 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad404cc45a74432b83dcc1836771b66'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:13.771334 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1446/2871] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:40:13 2022] DELETE /identity/v3/users/2ad404cc45a74432b83dcc1836771b66 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:40:13.776299 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.776645 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.776989 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.798384 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:13.800521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:13.860503 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:13.876038 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1426/2872] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:40:13.886313 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:13.888007 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:13.958508 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb567c-d546-4ead-9acf-d0b9d2b86aa7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:13.962557 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c7471-ae9b-4cfd-81b6-f00bcbc7afab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:13.963287 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7471-ae9b-4cfd-81b6-f00bcbc7afab service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:13.963578 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7471-ae9b-4cfd-81b6-f00bcbc7afab service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:13.963857 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7471-ae9b-4cfd-81b6-f00bcbc7afab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:13.987008 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c7471-ae9b-4cfd-81b6-f00bcbc7afab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:13.989867 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c7471-ae9b-4cfd-81b6-f00bcbc7afab service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.002057 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1447/2873] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:40:14.332163 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.333761 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:14.352427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036018e9-9fc5-4ba1-a73e-f31ffaae17d8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.367669 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.369486 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.370015 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.370326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] PATH_INFO: `/v3/projects/58c4e094a3694449a93f7d39aee0187e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.372135 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c4e094a3694449a93f7d39aee0187e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:14.374981 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.433730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c4e094a3694449a93f7d39aee0187e', 'request_id': 'req-7cada751-df8c-4d36-9c2d-02bf4cc90da6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:14.441852 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c4e094a3694449a93f7d39aee0187e', 'request_id': 'req-7cada751-df8c-4d36-9c2d-02bf4cc90da6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:14.449344 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c4e094a3694449a93f7d39aee0187e', 'request_id': 'req-7cada751-df8c-4d36-9c2d-02bf4cc90da6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:14.466362 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] The token cache is being invalidate because project 58c4e094a3694449a93f7d39aee0187e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:40:14.467328 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:14.468030 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7cada751-df8c-4d36-9c2d-02bf4cc90da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:40:14.469754 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1427/2874] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:40:14 2022] DELETE /identity/v3/projects/58c4e094a3694449a93f7d39aee0187e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:40:14.475766 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.476199 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.476594 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.498701 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:14.500015 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:14.579266 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1448/2875] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:40:14.588221 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.588564 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.588887 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.611266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:14.612594 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:14.621790 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1428/2876] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:40:14.626797 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.637727 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67124f-bd4d-4d61-a293-7505bb8f79e6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.642951 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.643629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.643943 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.644407 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.645018 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:40:14.645337 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:14.647080 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.662771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1449/2877] 10.222.0.30 () {58 vars in 1152 bytes} [Thu Jun 16 18:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:40:14.667680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.681950 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c9749-893d-4cca-9922-38d8c7eae2d8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.691296 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.692537 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.693056 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.693569 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.694897 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:14.698722 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.739491 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1429/2878] 10.222.0.30 () {60 vars in 1152 bytes} [Thu Jun 16 18:40:14 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:40:14.744456 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.756774 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627f4a4-b450-475e-83b1-5531d003bc5e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.762751 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:14.836677 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.837936 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.838455 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.838973 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.839997 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:14.842611 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.868623 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1450/2879] 10.222.0.30 () {60 vars in 1143 bytes} [Thu Jun 16 18:40:14 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:40:14.873800 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.886266 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3cca-d7b0-4883-a643-638f3b7b002a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.893183 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.893931 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.894225 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.894515 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.895351 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:14.897285 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.917473 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1430/2880] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:40:14.922647 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.934693 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b107dd7-a40e-4ce1-a00d-1b670b4e2069 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.940631 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac51e61-2ff8-4e85-b2d7-85b624313e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.941341 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac51e61-2ff8-4e85-b2d7-85b624313e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.941626 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac51e61-2ff8-4e85-b2d7-85b624313e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.941992 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac51e61-2ff8-4e85-b2d7-85b624313e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:14.943137 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac51e61-2ff8-4e85-b2d7-85b624313e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:14.944969 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac51e61-2ff8-4e85-b2d7-85b624313e79 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:14.957566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1451/2881] 10.222.0.30 () {58 vars in 1121 bytes} [Thu Jun 16 18:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:40:14.962365 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:14.973776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e993f0d-c6a4-4dc3-9b05-5fd99d025db3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:14.979421 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:14.980464 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:14.980834 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:14.981168 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] PATH_INFO: `/v3/projects/3bc7c74836e6485cbb5799b77faee4fd/users/006d66ec6573443bb268408501c0aaf3/roles/f21b9c85de034e5fac7761764c96d2f6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:15.001440 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc7c74836e6485cbb5799b77faee4fd, user_id=006d66ec6573443bb268408501c0aaf3, role_id=f21b9c85de034e5fac7761764c96d2f6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:15.004521 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:15.020774 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1431/2882] 10.222.0.30 () {60 vars in 1482 bytes} [Thu Jun 16 18:40:14 2022] PUT /identity/v3/projects/3bc7c74836e6485cbb5799b77faee4fd/users/006d66ec6573443bb268408501c0aaf3/roles/f21b9c85de034e5fac7761764c96d2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:40:15.026502 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55613e-9aad-4ffc-975f-1b2a62c8d609 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:15.026901 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55613e-9aad-4ffc-975f-1b2a62c8d609 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:15.027228 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55613e-9aad-4ffc-975f-1b2a62c8d609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:15.049386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9b55613e-9aad-4ffc-975f-1b2a62c8d609 None None] MFA Rules not processed for user `bd8fd8fbb8b9467bac357373688c6910`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:15.050836 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9b55613e-9aad-4ffc-975f-1b2a62c8d609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:15.142035 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1452/2883] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:40:15.153336 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:15.155773 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:15.227286 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d70723-3a77-44a9-b3f1-447bebd9300a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:15.231237 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:15.232025 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:15.232497 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:15.232830 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:15.243711 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:15.245251 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:15.257905 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1432/2884] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:40:16.202345 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a38b-9828-4490-9a34-5d1d908a5657 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:16.203147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a38b-9828-4490-9a34-5d1d908a5657 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:16.203147 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a38b-9828-4490-9a34-5d1d908a5657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:16.241181 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1c34a38b-9828-4490-9a34-5d1d908a5657 None None] MFA Rules not processed for user `006d66ec6573443bb268408501c0aaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:16.242615 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1c34a38b-9828-4490-9a34-5d1d908a5657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:16.344321 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1453/2885] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:40:16.358772 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:40:16.360636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:16.392964 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:16.466326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b71673-c70c-4c98-8241-7f66689472d7 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:40:16.473132 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cffaa8-3559-426f-9e9f-38e4d3298c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:40:16.474176 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cffaa8-3559-426f-9e9f-38e4d3298c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:16.474605 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cffaa8-3559-426f-9e9f-38e4d3298c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:16.475034 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cffaa8-3559-426f-9e9f-38e4d3298c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:16.490335 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cffaa8-3559-426f-9e9f-38e4d3298c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:40:16.492522 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cffaa8-3559-426f-9e9f-38e4d3298c10 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:40:16.506662 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1433/2886] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:40:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:40:17.978592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2ac9d-102f-483c-9486-d9d29c7d4ca0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:40:17.978592 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2ac9d-102f-483c-9486-d9d29c7d4ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:40:17.979469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2ac9d-102f-483c-9486-d9d29c7d4ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:40:17.999422 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3dd2ac9d-102f-483c-9486-d9d29c7d4ca0 None None] MFA Rules not processed for user `006d66ec6573443bb268408501c0aaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:40:18.000725 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3dd2ac9d-102f-483c-9486-d9d29c7d4ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:40:18.008457 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1454/2887] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:40:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:40:26.500435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=19 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:40:27.993100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.841075 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.841993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.841993 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.875062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] MFA Rules not processed for user `006d66ec6573443bb268408501c0aaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:20.876352 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:20.907447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1434/2888] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:41:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:41:30.904820 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=15 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:42:11.746799 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:42:11.747643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:42:11.747643 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:42:11.769956 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] MFA Rules not processed for user `e5cbf8d443de4a9fb627ef316fbd033f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:42:11.771343 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:42:11.874796 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1455/2889] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:42:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:42:15.233345 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:42:15.240319 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:42:15.317267 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:42:15.419492 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776b774-c56d-4005-8645-a8ebc966e856 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:42:15.428777 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:42:15.430326 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:42:15.430806 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:42:15.431580 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] PATH_INFO: `/v3/users/006d66ec6573443bb268408501c0aaf3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:42:15.434567 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=006d66ec6573443bb268408501c0aaf3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:42:15.437458 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:42:15.475199 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7068e-92b1-427e-836d-750c76f90155 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:42:15.482435 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:42:15.483312 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:42:15.483602 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:42:15.484107 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:42:15.487500 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:42:15.507861 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:42:15.509748 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:42:15.534181 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1435/2890] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:42:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:42:15.580235 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006d66ec6573443bb268408501c0aaf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:15.608236 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006d66ec6573443bb268408501c0aaf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:15.625697 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006d66ec6573443bb268408501c0aaf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:15.627805 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006d66ec6573443bb268408501c0aaf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:15.628079 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006d66ec6573443bb268408501c0aaf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:15.655775 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1456/2891] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:42:15 2022] DELETE /identity/v3/users/006d66ec6573443bb268408501c0aaf3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:42:15.959458 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:42:15.994516 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922852e0-916f-4d1e-98a5-afe7431024c6 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:42:16.093916 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:42:16.095024 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:42:16.095331 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:42:16.095593 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] PATH_INFO: `/v3/projects/3bc7c74836e6485cbb5799b77faee4fd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:42:16.097017 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc7c74836e6485cbb5799b77faee4fd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:42:16.099093 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:42:16.170568 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc7c74836e6485cbb5799b77faee4fd', 'request_id': 'req-b307fbe0-0e44-4d57-a7d3-250cd740032d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:16.178747 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:42:16.180644 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc7c74836e6485cbb5799b77faee4fd', 'request_id': 'req-b307fbe0-0e44-4d57-a7d3-250cd740032d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:16.190054 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc7c74836e6485cbb5799b77faee4fd', 'request_id': 'req-b307fbe0-0e44-4d57-a7d3-250cd740032d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:16.227123 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] The token cache is being invalidate because project 3bc7c74836e6485cbb5799b77faee4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:42:16.227578 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:16.228406 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b307fbe0-0e44-4d57-a7d3-250cd740032d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:42:16.230801 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1436/2892] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:42:15 2022] DELETE /identity/v3/projects/3bc7c74836e6485cbb5799b77faee4fd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:42:24.366656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:42:24.369094 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:42:24.457837 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:42:24.488273 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3718e4-72e5-45b2-b2b6-216dca370e5e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:42:24.583744 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:42:24.584398 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:42:24.584709 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:42:24.584963 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:42:24.587043 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:42:24.601231 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:42:24.604813 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:42:24.622902 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1457/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:42:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:42:26.223970 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=10,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:42:34.613539 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=33 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:09.565280 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:09.566210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:09.566210 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:09.589186 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] MFA Rules not processed for user `e5cbf8d443de4a9fb627ef316fbd033f`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:44:09.590477 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:09.678404 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1437/2894] 10.222.0.30 () {58 vars in 957 bytes} [Thu Jun 16 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:44:09.726673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:09.728447 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:09.830233 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a6a86-32be-47ec-99c8-03d115127c10 service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:09.836396 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49b361e969094db296e3fc02eda18dc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:09.837100 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:09.837386 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:09.837663 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:09.839710 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:09.966460 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:09.967845 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:09.995286 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1458/2895] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:44:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:44:10.868326 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:10.871702 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:10.995338 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760dcec-27bd-4d8f-a012-6f082300f489 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:10.999287 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:11.000291 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:11.000481 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:11.000625 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:11.004939 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:11.021938 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:11.024447 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:11.042592 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1438/2896] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:44:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:44:19.991438 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=75 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:21.032339 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=60 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:44:25.119070 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:44:25.121353 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:25.145085 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e7c95-0d7c-4346-b4ca-192597ddf24b service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:44:25.150507 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:44:25.151665 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:44:25.152245 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:44:25.152803 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:44:25.156842 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:44:25.174673 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:44:25.177953 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:44:25.192517 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1459/2897] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:44:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:44:35.187743 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:45:58.661680 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:45:58.671943 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e1d9-cd7c-4512-8acd-3337c956cc07 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:45:58.676022 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:45:58.676983 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:45:58.677808 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:45:58.678096 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:45:58.680172 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:45:58.692502 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:45:58.695730 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:45:58.716137 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1439/2898] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:45:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:46:00.292019 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:00.296394 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:00.335587 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70217cbb-ea93-459b-9579-881193437a39 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:00.346792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:00.347899 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:00.350327 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:00.350884 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:00.354758 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:00.381234 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:00.384347 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:00.424023 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1460/2899] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:46:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:46:02.835485 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:02.838656 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:02.869259 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f858ef-99dd-4c9d-9aa4-69685d64eba0 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:02.876726 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793065092530407983e5da2446d2e939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:02.877451 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:02.877751 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:02.878032 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:02.881723 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:02.902737 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:02.905218 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:02.931867 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1440/2900] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:46:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:46:10.402435 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:12.913776 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=12 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:13.875087 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:13.885965 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d3b7e-0f1e-4c57-a49a-8d0f6582b8e7 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:13.890033 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558f4050abe8467aa527d73a49f101b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:13.890801 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:13.891088 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:13.891370 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:13.893335 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:13.975400 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:13.976921 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:13.988110 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1461/2901] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:46:22.635924 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:22.638297 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:22.687238 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a256-a607-4358-b303-a9144bf58c67 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:22.697066 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:22.698230 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.698729 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.699236 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.702858 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:22.719933 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:22.722803 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:22.742590 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:22.744925 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:22.746640 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1441/2902] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:46:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:46:22.765601 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aea89c-c378-448e-8a97-0cb4b17a0eb8 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:22.777254 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:22.778765 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:22.779117 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:22.779476 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] PATH_INFO: `/v3/users/e5cbf8d443de4a9fb627ef316fbd033f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:22.781294 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5cbf8d443de4a9fb627ef316fbd033f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:22.783711 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:22.877656 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5cbf8d443de4a9fb627ef316fbd033f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:22.906995 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5cbf8d443de4a9fb627ef316fbd033f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:22.918042 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5cbf8d443de4a9fb627ef316fbd033f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:22.918972 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5cbf8d443de4a9fb627ef316fbd033f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:22.919704 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5cbf8d443de4a9fb627ef316fbd033f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:22.941048 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1462/2903] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:46:22 2022] DELETE /identity/v3/users/e5cbf8d443de4a9fb627ef316fbd033f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:23.201857 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.226643 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc6171-02a4-4a20-9b65-2c97b6ac4077 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.309243 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.310143 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.310427 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.310699 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] PATH_INFO: `/v3/projects/dff4784f6ab3499b88cb38c81780b754` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.312065 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff4784f6ab3499b88cb38c81780b754)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.314062 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.374571 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff4784f6ab3499b88cb38c81780b754', 'request_id': 'req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.382884 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff4784f6ab3499b88cb38c81780b754', 'request_id': 'req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.390682 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff4784f6ab3499b88cb38c81780b754', 'request_id': 'req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.407329 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] The token cache is being invalidate because project dff4784f6ab3499b88cb38c81780b754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:46:23.407657 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.408364 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.410016 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1442/2904] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:46:23 2022] DELETE /identity/v3/projects/dff4784f6ab3499b88cb38c81780b754 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:23.416753 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.418493 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:23.433243 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf4540-94d3-4e7b-ab59-d5ae43e2913d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.506469 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.507929 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.508475 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.508964 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] PATH_INFO: `/v3/users/53d69d0d5b6847189d3f84975aad6742` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.510824 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d69d0d5b6847189d3f84975aad6742)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.513682 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:23.522596 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:23.581016 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d69d0d5b6847189d3f84975aad6742'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.597515 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d69d0d5b6847189d3f84975aad6742'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.607139 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d69d0d5b6847189d3f84975aad6742'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.608500 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d69d0d5b6847189d3f84975aad6742'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.609835 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d69d0d5b6847189d3f84975aad6742'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:23.627423 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1463/2905] 10.222.0.30 () {58 vars in 1223 bytes} [Thu Jun 16 18:46:23 2022] DELETE /identity/v3/users/53d69d0d5b6847189d3f84975aad6742 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:46:23.878059 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:46:23.880784 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:46:23.897141 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478f20f-8193-49b7-8b20-e91d2aa1ad6a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:46:23.978636 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd8fd8fbb8b9467bac357373688c6910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243115c739b84006986ba2e6345f2509', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:46:23.979635 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:46:23.979961 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:46:23.980418 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] PATH_INFO: `/v3/projects/53b2dc9e01c9411f8253637fb8439dbd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:46:23.981923 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b2dc9e01c9411f8253637fb8439dbd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:46:23.983962 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:46:24.009631 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:24.045932 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b2dc9e01c9411f8253637fb8439dbd', 'request_id': 'req-b8865da6-47a0-4954-9e1a-b811d20d84e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:24.054273 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b2dc9e01c9411f8253637fb8439dbd', 'request_id': 'req-b8865da6-47a0-4954-9e1a-b811d20d84e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:24.066817 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b2dc9e01c9411f8253637fb8439dbd', 'request_id': 'req-b8865da6-47a0-4954-9e1a-b811d20d84e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:24.090831 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] The token cache is being invalidate because project 53b2dc9e01c9411f8253637fb8439dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:46:24.091343 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:24.092489 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b8865da6-47a0-4954-9e1a-b811d20d84e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:46:24.095262 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1443/2906] 10.222.0.30 () {58 vars in 1232 bytes} [Thu Jun 16 18:46:23 2022] DELETE /identity/v3/projects/53b2dc9e01c9411f8253637fb8439dbd => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:46:33.623403 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:46:34.083659 dv-cinder-813229-10-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:30.346629 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:30.349771 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:30.469490 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9167-40a8-403f-829b-1c3e5924415b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:30.549115 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad094d850d946d18b683bfc04c6f3e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0fabfeb40ce4613a56f73db68fde3ec', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:30.549792 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:30.550071 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:30.550566 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:30.552691 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:30.565844 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:30.567826 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c494dfa-1fb9-42e0-a88b-a783fc2a7232 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:30.579961 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1464/2907] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:40.574942 dv-cinder-813229-10-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=58 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}